-- Logs begin at Tue 2022-08-23 07:22:16 UTC, end at Tue 2022-08-23 08:45:19 UTC. -- Aug 23 07:54:04.980327 dv-cinder-854070-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 23 07:54:04.989618 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 23 07:54:04 2022] *** Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: nodename: dv-cinder-854070-4-iscsi Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: machine: x86_64 Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: clock source: unix Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: pcre jit disabled Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: detected number of CPU cores: 4 Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: current working directory: / Aug 23 07:54:04.991492 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: detected binary path: /usr/bin/uwsgi-core Aug 23 07:54:04.992075 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: your processes number limit is 47680 Aug 23 07:54:04.992075 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: your memory page size is 4096 bytes Aug 23 07:54:04.992075 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: detected max file descriptor number: 2048 Aug 23 07:54:04.992075 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: lock engine: pthread robust mutexes Aug 23 07:54:04.992075 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: thunder lock: enabled Aug 23 07:54:04.992075 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 23 07:54:04.992075 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 23 07:54:05.031047 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: Python main interpreter initialized at 0x555d792eb9c0 Aug 23 07:54:05.031047 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: python threads support enabled Aug 23 07:54:05.031047 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: your server socket listen backlog is limited to 100 connections Aug 23 07:54:05.031047 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: your mercy for graceful operations on workers is 90 seconds Aug 23 07:54:05.031357 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: mapped 403077 bytes (393 KB) for 2 cores Aug 23 07:54:05.031432 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: *** Operational MODE: preforking *** Aug 23 07:54:05.031664 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: *** uWSGI is running in multiple interpreter mode *** Aug 23 07:54:05.031664 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: spawned uWSGI master process (pid: 81596) Aug 23 07:54:05.031852 dv-cinder-854070-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 23 07:54:05.033212 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: spawned uWSGI worker 1 (pid: 81606, cores: 1) Aug 23 07:54:05.033212 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: spawned uWSGI worker 2 (pid: 81607, cores: 1) Aug 23 07:54:05.033212 dv-cinder-854070-4-iscsi devstack@keystone.service[81596]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 23 07:54:09.359601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 07:54:09.359601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: from cryptography.utils import int_from_bytes Aug 23 07:54:09.360418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 07:54:09.360418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: from cryptography.utils import int_from_bytes Aug 23 07:54:09.362125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 07:54:09.362125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: from cryptography.utils import int_from_bytes Aug 23 07:54:09.362588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 07:54:09.362588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: from cryptography.utils import int_from_bytes Aug 23 07:54:09.411476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 07:54:09.411679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 07:54:09.411841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] command line args: [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 07:54:09.412108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 07:54:09.412284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ================================================================================ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 07:54:09.412533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] admin_token = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.412778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] config_dir = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.412965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.413148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] config_source = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.413329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] debug = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.413524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.413787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 07:54:09.413916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.413995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 07:54:09.414062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.414121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] command line args: [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 07:54:09.414301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.414389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 07:54:09.414539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ================================================================================ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 07:54:09.414617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.414766 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.414849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] admin_token = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.414978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_config_append = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_dir = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_dir = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_source = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_file = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] debug = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.415916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.416950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.417100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.417177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_config_append = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.417297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.417386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.417386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_dir = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.417570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.417644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.417850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.418923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.419042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.419168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.419869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.419968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] publish_errors = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.420975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.421057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.421194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.421272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.421428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] standard_threads = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.421527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.421659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.421886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] publish_errors = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_journal = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_json = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_stderr = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] use_syslog = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.422969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.423166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.423243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.423407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.423486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.423644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] standard_threads = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.423729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.423880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.423964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.424127 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.424306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.424306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.424462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.424541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_journal = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.424618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.424767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_json = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.424848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_stderr = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.424919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.424994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_syslog = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.425065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.425188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 07:54:09.425279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.425448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.425526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.425691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.425774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.425952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426902 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.426999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.427178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.433620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.433620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.433941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.433941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.434989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.435936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.436951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.437946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.438980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.439954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.440888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441643 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.441994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442454 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.442996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.443949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.444990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.445932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.446924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.447995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.external = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.448920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.password = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449227 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] auth.token = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.449930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.external = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.450918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.password = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.token = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.451977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.452134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.452215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.452384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.452464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.452634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.452714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.452873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.452955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.453117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.453203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.453368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.453456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.453621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.453707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.453860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.453940 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.454100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.454180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.454346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.454430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.454585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.454663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.454814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.454892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.455084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.455169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.455322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.455402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.455574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.455654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.455809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.455890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.456063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.456142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.456299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.456424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.456547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.456629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.456783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.456863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.457993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.458073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.458223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.458301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.458460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.458539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.458709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.458802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459137 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.459978 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.460144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.460226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.460390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.460476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.460631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.460711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.460867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.460948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.461118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.461198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.461353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.461504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.461583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.461655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.461809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.461888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.462046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.462134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.462277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.462357 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.462513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.462599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.462748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.462824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463253 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.463973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.464051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.464209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.464296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.464469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.464550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.464683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.464763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.464913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.465929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.466621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.466738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.466876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.467967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.468137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.468215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.468382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.468469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.468615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.468693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.468853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.468932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.469085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.469171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.469318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.469403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.469547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.469628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.469786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.469874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.470024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.470102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.470260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.470338 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.470505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.470594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.470752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.470831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471004 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.471941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.472958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.473122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.473203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.473363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.473450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.473603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.473689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.473831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.473911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.474077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.474166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.474314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.474392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.474546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.474624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.474780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.474868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.475035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.475117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.475281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.475387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.475528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.475632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.475763 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.475887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.476015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.476140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.476227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.476669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.476757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.476922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477660 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.477984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.478128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.478208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.478363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.478444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.478601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.478752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.478849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.478928 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.479119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.479199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.479354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.479446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.479602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.479682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.479834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.479912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.480071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.480159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.480313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.480392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.480546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.480624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.480784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.480871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.481018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] role.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.481096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.481263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] role.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.481343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.481494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.481582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.481783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.482354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.482529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.482610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.482774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.482861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.483047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.483131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.483289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.483367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.483531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.483623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.483775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.483855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.484973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.485060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.485216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.485295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.485449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.485608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.485687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.485848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.485929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.486098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.486177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.486348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.486436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.486597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.486677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.486847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.486925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.487104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.487197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.487365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.487452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.487609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.487687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.487845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.487931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.488077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.488157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.488308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.488387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.488538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.488625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.488787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.488946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.489027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.489189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.489268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.489443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.489524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.489697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.489777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.489931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.490008 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.490168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.490249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.490405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.490484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.490653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.490762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.490893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.491952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.492110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.492189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.492355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.492437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.492591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.492670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.492826 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.492905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.493061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.493142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.493293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.493371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.493535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.493615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.493760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.493840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.494000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.494080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.494242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.494320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.494501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.494582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.494750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.494840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.495963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.496995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.497150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.497228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.497422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.497510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.497669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.497814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.497934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498578 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.498998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.499099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81607) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 07:54:09.499244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.499487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.499729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.499970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.500220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.500463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.500711 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.500953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.501204 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.501445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.501694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.501928 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.502177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.502417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 07:54:09.502591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 07:54:09.532274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.532496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.532719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.533425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.533661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.533893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.540990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.542285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.572957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.573180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.573454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.573694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.626501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.626976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.675020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.675253 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.677375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.677613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.836296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.836524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.836652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.836740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.836887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.836975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.837115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.837200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.837340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.837433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.837562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.837782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.871727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.871875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.871967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.872930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.873055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.873149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.884077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.884448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.897170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.897396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.897619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.897705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.897842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.897927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.898059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.898143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.898272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.898358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.898483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81607) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.898567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.898701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.898921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 07:54:09.915684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.915898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 07:54:09.916235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.916318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.916435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 07:54:09.916521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 07:54:09.916709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.916792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.916973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.917094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 07:54:09.917249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.917474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.917556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.917738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.918021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.918290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.927127 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.927407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.927677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.927925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.928008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.930101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.930101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.930101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.934930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.935901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.937864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.938149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.938436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.938723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.938877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.939035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.939276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.939359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.939563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.939644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.939842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.939952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.940146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.940228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.940413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.940494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.940692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.940776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.940972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.941359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.941666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.941941 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.964205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.965661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.965838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.965951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.967923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.970256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.971235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.971522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.971808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.973001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.973295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.973585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.978185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.978472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.980111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.980402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.982627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.982918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.983225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.983511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.983789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.984067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.984341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.984612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.984726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:09.984907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.985195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.985483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.985767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.986051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.986328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:09.986601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.001945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.002227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.004217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.004503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.006147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.010372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.010462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.010462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.011788 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.012092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.014418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.016890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.017596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.020098 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.020844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.021147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.021438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.021721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.022030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.022299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.023398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.023710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.024002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.024307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.024599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.024906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.038826 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.039144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.039416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.039682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.041577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.041875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.042148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.042414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.046623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.048285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.048455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.056561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.056869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.056869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.064061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.064345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.064630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.064913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.065196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.065474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.065762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.067133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.067426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.067711 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.067997 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.068284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.068571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.068885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.082136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.085190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.085442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.088125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.088583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.089568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.091670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.091967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.092325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.092395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.093562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.095084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.095385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.095687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.098754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.100239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.102370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.103186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.103478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.103760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.103894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.104038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.106957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.107256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.107546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.107834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.112030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.112321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.112602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.115988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.116288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.116577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.119555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81607) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.119838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.120117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.120413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.120668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.120949 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.121221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.121490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.121768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.122044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.122349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.122612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.122892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.123174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81607) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.123683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 07:54:10.123976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.124263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.124544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.124831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.125112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.125387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.125659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.125942 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.126254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.126534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.126811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.127107 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.127375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 07:54:10.158215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555d792eb9c0 pid: 81607 (default app) Aug 23 07:54:10.161567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0baa71-e42a-4baa-ab85-af2aa646339c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:10.161848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0baa71-e42a-4baa-ab85-af2aa646339c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:10.162117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0baa71-e42a-4baa-ab85-af2aa646339c None None] PATH_INFO: `/v3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:10.162840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555d792eb9c0 pid: 81606 (default app) Aug 23 07:54:10.166220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Tue Aug 23 07:54:10 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:17.745117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d0994-3132-4327-bd59-e1de20450a07 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:17.745630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d0994-3132-4327-bd59-e1de20450a07 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:17.745712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d0994-3132-4327-bd59-e1de20450a07 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:17.750449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:17.756885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:17.757287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:17.757421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:17.827994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: INFO dbcounter [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] Registered counter for database keystone Aug 23 07:54:17.848566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG oslo_db.sqlalchemy.engines [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81607) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 07:54:17.854644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writer thread running {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 07:54:18.034332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG passlib.handlers.bcrypt [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81607) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 07:54:18.034671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG passlib.handlers.bcrypt [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81607) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 07:54:18.089450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:18.091051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:18.216930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 23 07:54:18.657474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d0994-3132-4327-bd59-e1de20450a07 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:18.726904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: INFO dbcounter [None req-047d0994-3132-4327-bd59-e1de20450a07 None None] Registered counter for database keystone Aug 23 07:54:18.746933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG oslo_db.sqlalchemy.engines [None req-047d0994-3132-4327-bd59-e1de20450a07 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81606) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 07:54:18.753220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writer thread running {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 07:54:18.874560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d0994-3132-4327-bd59-e1de20450a07 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:18.878882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f43437-6b8d-421d-8db9-333958879c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:18.879568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f43437-6b8d-421d-8db9-333958879c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:18.879845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f43437-6b8d-421d-8db9-333958879c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:18.880118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f43437-6b8d-421d-8db9-333958879c33 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:18.888403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f43437-6b8d-421d-8db9-333958879c33 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:18.941162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING keystone.common.rbac_enforcer.enforcer [None req-52f43437-6b8d-421d-8db9-333958879c33 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 07:54:18.941501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f43437-6b8d-421d-8db9-333958879c33 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:18.951916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Tue Aug 23 07:54:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 07:54:18.957789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:18.972970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062e344-ca9e-4f94-9b26-188b15f4fc43 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:18.977429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1c808-b0ee-46d3-9e5e-8390bcfaa2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:18.978594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1c808-b0ee-46d3-9e5e-8390bcfaa2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:18.978910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1c808-b0ee-46d3-9e5e-8390bcfaa2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:18.979661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1c808-b0ee-46d3-9e5e-8390bcfaa2f2 admin admin] PATH_INFO: `/v3/projects/ee2616c3f460471aaf732aeea2d3bd15` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:18.981224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1c808-b0ee-46d3-9e5e-8390bcfaa2f2 admin admin] RBAC: Authorizing `identity:get_project(project_id=ee2616c3f460471aaf732aeea2d3bd15)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:19.034754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING keystone.common.rbac_enforcer.enforcer [None req-93f1c808-b0ee-46d3-9e5e-8390bcfaa2f2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 07:54:19.035129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1c808-b0ee-46d3-9e5e-8390bcfaa2f2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:19.039662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Tue Aug 23 07:54:18 2022] GET /identity/v3/projects/ee2616c3f460471aaf732aeea2d3bd15 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:20.886799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07175694-8435-4052-8bec-3294b4498fbf None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:20.887352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07175694-8435-4052-8bec-3294b4498fbf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:20.887469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07175694-8435-4052-8bec-3294b4498fbf None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:20.892516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:20.897832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bcea-a564-4f0c-87c7-78e9d5fb513d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:20.898266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bcea-a564-4f0c-87c7-78e9d5fb513d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:20.898360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bcea-a564-4f0c-87c7-78e9d5fb513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:20.917736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-09c8bcea-a564-4f0c-87c7-78e9d5fb513d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:20.919347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-09c8bcea-a564-4f0c-87c7-78e9d5fb513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:20.927028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 07:54:21.200412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07175694-8435-4052-8bec-3294b4498fbf None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:21.212561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07175694-8435-4052-8bec-3294b4498fbf None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:21.216774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab08be-1b41-43a4-b425-d81c6d11fd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:21.217369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab08be-1b41-43a4-b425-d81c6d11fd00 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:21.217649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab08be-1b41-43a4-b425-d81c6d11fd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:21.217919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab08be-1b41-43a4-b425-d81c6d11fd00 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:21.225481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab08be-1b41-43a4-b425-d81c6d11fd00 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:21.227282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab08be-1b41-43a4-b425-d81c6d11fd00 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:21.236450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Tue Aug 23 07:54:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 07:54:21.241583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bcea-a564-4f0c-87c7-78e9d5fb513d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:21.251327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bcea-a564-4f0c-87c7-78e9d5fb513d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:21.255758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de2e60-2a46-40e0-a8ad-47b3fddcb061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:21.256375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de2e60-2a46-40e0-a8ad-47b3fddcb061 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:21.256657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de2e60-2a46-40e0-a8ad-47b3fddcb061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:21.256932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de2e60-2a46-40e0-a8ad-47b3fddcb061 admin admin] PATH_INFO: `/v3/users/70cd347e5af341fcb03bb3178f2c4247` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:21.258248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de2e60-2a46-40e0-a8ad-47b3fddcb061 admin admin] RBAC: Authorizing `identity:get_user(user_id=70cd347e5af341fcb03bb3178f2c4247)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:21.260104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de2e60-2a46-40e0-a8ad-47b3fddcb061 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:21.261621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Tue Aug 23 07:54:21 2022] GET /identity/v3/users/70cd347e5af341fcb03bb3178f2c4247 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:23.223489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213451cc-6a36-4e6b-b2e7-8566309206d2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:23.224022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213451cc-6a36-4e6b-b2e7-8566309206d2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:23.224022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213451cc-6a36-4e6b-b2e7-8566309206d2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:23.225369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:23.230665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c836e8-0c36-4ccc-9127-5696d5c3f626 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:23.231125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c836e8-0c36-4ccc-9127-5696d5c3f626 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:23.231230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78486523-077e-478e-8214-6ac793db4dd9 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:23.231320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c836e8-0c36-4ccc-9127-5696d5c3f626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:23.231405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78486523-077e-478e-8214-6ac793db4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:23.231613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78486523-077e-478e-8214-6ac793db4dd9 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:23.233410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:23.238431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:23.238715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:23.239043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:23.250121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-32c836e8-0c36-4ccc-9127-5696d5c3f626 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:23.251320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-32c836e8-0c36-4ccc-9127-5696d5c3f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:23.281068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG passlib.handlers.bcrypt [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81606) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 07:54:23.281492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG passlib.handlers.bcrypt [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81606) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 07:54:23.327171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:23.328753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:23.331687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:23.426414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:54:23.594933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c836e8-0c36-4ccc-9127-5696d5c3f626 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:23.617889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:23.694148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:23.697614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c836e8-0c36-4ccc-9127-5696d5c3f626 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:23.730425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291269f7-ec6f-4978-9dad-8d28d5076da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:23.731057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291269f7-ec6f-4978-9dad-8d28d5076da3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:23.731338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291269f7-ec6f-4978-9dad-8d28d5076da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:23.731611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291269f7-ec6f-4978-9dad-8d28d5076da3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:23.796255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a72ef-82f4-4daa-82cb-fa939bbedf36 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:23.798594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:23.801440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291269f7-ec6f-4978-9dad-8d28d5076da3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:23.802914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291269f7-ec6f-4978-9dad-8d28d5076da3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:23.825693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:23.834659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed0b01-9f16-438d-886f-1067e6e14fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:23.835819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed0b01-9f16-438d-886f-1067e6e14fba None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:23.836153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed0b01-9f16-438d-886f-1067e6e14fba None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:23.836434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed0b01-9f16-438d-886f-1067e6e14fba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:23.848234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed0b01-9f16-438d-886f-1067e6e14fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c1ed0b01-9f16-438d-886f-1067e6e14fba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:23.857090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:23.861018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:23.861826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 8/14] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:54:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:23.864163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed0b01-9f16-438d-886f-1067e6e14fba None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:23.902058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 7/15] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:54:23.907750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291269f7-ec6f-4978-9dad-8d28d5076da3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:23.964797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed0b01-9f16-438d-886f-1067e6e14fba None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:23.998687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:23.999327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:23.999608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:23.999884 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:24.000547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:24.000849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:24.002294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:24.005272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291269f7-ec6f-4978-9dad-8d28d5076da3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:24.014269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 9/16] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:24.019912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:24.027375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:24.039076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:24.043172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:24.043172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:24.043172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:24.049532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:24.051286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:24.052739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:54:24.072487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:24.073432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:54:24.074238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:54:24.074238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:54:24.074238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:24.074238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 8/17] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:24.079509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:24.121623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f073c-8e07-4ea6-99c2-eb52786821a1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:24.155417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe76a5-666c-4c2d-8b65-b97a0dd58bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:24.156035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe76a5-666c-4c2d-8b65-b97a0dd58bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:24.156311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe76a5-666c-4c2d-8b65-b97a0dd58bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:24.156579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe76a5-666c-4c2d-8b65-b97a0dd58bb7 None admin] PATH_INFO: `/v3/users/70cd347e5af341fcb03bb3178f2c4247` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:24.158313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe76a5-666c-4c2d-8b65-b97a0dd58bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=70cd347e5af341fcb03bb3178f2c4247)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:24.159805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe76a5-666c-4c2d-8b65-b97a0dd58bb7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:24.161364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 10/18] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:24 2022] GET /identity/v3/users/70cd347e5af341fcb03bb3178f2c4247 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:24.166962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe76a5-666c-4c2d-8b65-b97a0dd58bb7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:24.179086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbae72-dde6-43b4-b3d3-f26cc7367418 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:24.215135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:24.215744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:24.216025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:24.216294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:24.216931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:24.217231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:24.218621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:24.236530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 9/19] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:24.263235 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:24.270261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe76a5-666c-4c2d-8b65-b97a0dd58bb7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:24.309003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345693c-4e8c-4bac-9abe-30588d5752f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:24.309651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345693c-4e8c-4bac-9abe-30588d5752f6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:24.309924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345693c-4e8c-4bac-9abe-30588d5752f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:24.310191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345693c-4e8c-4bac-9abe-30588d5752f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:24.312045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345693c-4e8c-4bac-9abe-30588d5752f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:24.313518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345693c-4e8c-4bac-9abe-30588d5752f6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:24.315277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 11/20] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:24.321506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:24.358437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:24.426475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b10be-2e95-489d-9d7e-5fb4cdcce2cc None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:24.462828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39975be-df9b-4ca8-be32-499bdf1a0186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:24.463465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39975be-df9b-4ca8-be32-499bdf1a0186 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:24.463742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39975be-df9b-4ca8-be32-499bdf1a0186 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:24.464013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39975be-df9b-4ca8-be32-499bdf1a0186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:24.464742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39975be-df9b-4ca8-be32-499bdf1a0186 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.domain.id=default, user.id=70cd347e5af341fcb03bb3178f2c4247) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:24.465049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39975be-df9b-4ca8-be32-499bdf1a0186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:24.466483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39975be-df9b-4ca8-be32-499bdf1a0186 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:24.473917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 10/21] 10.222.0.58 () {56 vars in 1322 bytes} [Tue Aug 23 07:54:24 2022] GET /identity/v3/role_assignments?user.id=70cd347e5af341fcb03bb3178f2c4247&scope.domain.id=default&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:26.689208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcb64e-67b8-46f1-b7f3-3214bc63657b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:26.690232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcb64e-67b8-46f1-b7f3-3214bc63657b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:26.690712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcb64e-67b8-46f1-b7f3-3214bc63657b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:26.692626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 12/22] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:26.697804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899a4bd-11db-4c66-b0b4-103afc527e42 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:26.698596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899a4bd-11db-4c66-b0b4-103afc527e42 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:26.699105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899a4bd-11db-4c66-b0b4-103afc527e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:26.727929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9899a4bd-11db-4c66-b0b4-103afc527e42 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:26.729447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9899a4bd-11db-4c66-b0b4-103afc527e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:26.817439 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ad361-9da7-4610-8c9d-f6da8ce8176e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:26.818060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ad361-9da7-4610-8c9d-f6da8ce8176e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:26.818549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ad361-9da7-4610-8c9d-f6da8ce8176e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:26.824662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 11/24] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:26.825258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:26.834300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f8eb0-6cf1-42f0-b841-872493e3a68a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:26.834590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f8eb0-6cf1-42f0-b841-872493e3a68a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:26.834871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f8eb0-6cf1-42f0-b841-872493e3a68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:26.859960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f31f8eb0-6cf1-42f0-b841-872493e3a68a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:26.861149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f31f8eb0-6cf1-42f0-b841-872493e3a68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:26.891892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:26.942686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 12/25] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:27.075402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c763-749c-4a16-a515-86d044767d37 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.076082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c763-749c-4a16-a515-86d044767d37 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.076560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530c763-749c-4a16-a515-86d044767d37 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.078421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 14/26] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:27.083367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec72ce-5170-470f-a509-e360eb87fd66 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.083657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec72ce-5170-470f-a509-e360eb87fd66 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.083936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec72ce-5170-470f-a509-e360eb87fd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.107200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-71ec72ce-5170-470f-a509-e360eb87fd66 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:27.108959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-71ec72ce-5170-470f-a509-e360eb87fd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:27.139668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519f76-317c-45b2-b098-46f63098e671 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.140614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519f76-317c-45b2-b098-46f63098e671 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.141093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74519f76-317c-45b2-b098-46f63098e671 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.143865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 15/27] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:27.151082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043489f2-2882-4314-98b4-851bd677d0ac None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.151615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043489f2-2882-4314-98b4-851bd677d0ac None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.152089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043489f2-2882-4314-98b4-851bd677d0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.183236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-043489f2-2882-4314-98b4-851bd677d0ac None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:27.184445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-043489f2-2882-4314-98b4-851bd677d0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:27.218460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 13/28] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:27.224528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec72ce-5170-470f-a509-e360eb87fd66 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:27.313015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 16/29] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:54:27.374484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:27.390744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec72ce-5170-470f-a509-e360eb87fd66 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:27.398575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043489f2-2882-4314-98b4-851bd677d0ac None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:27.466383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d8633-eaa6-46c2-9ad0-c26150f06deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:27.467575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d8633-eaa6-46c2-9ad0-c26150f06deb None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.467861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d8633-eaa6-46c2-9ad0-c26150f06deb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.468133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d8633-eaa6-46c2-9ad0-c26150f06deb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.495633 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d8633-eaa6-46c2-9ad0-c26150f06deb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-904d8633-eaa6-46c2-9ad0-c26150f06deb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:27.503668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:27.504627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:27.505385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:27.508284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 14/30] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:27.509472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d8633-eaa6-46c2-9ad0-c26150f06deb None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:27.546271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:27.555956 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043489f2-2882-4314-98b4-851bd677d0ac None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:27.591303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:27.592827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.593323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.593810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.615350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:27.617114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:54:27.633731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:27.634613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:54:27.635462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:54:27.635462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:54:27.635462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:27.637297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 17/31] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:27.638295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:27.659911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d8633-eaa6-46c2-9ad0-c26150f06deb None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:27.695016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0657e-84f2-48b6-957f-ce8031b97c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:27.695656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0657e-84f2-48b6-957f-ce8031b97c27 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.696403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0657e-84f2-48b6-957f-ce8031b97c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.696866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0657e-84f2-48b6-957f-ce8031b97c27 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.697701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0657e-84f2-48b6-957f-ce8031b97c27 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:27.699309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0657e-84f2-48b6-957f-ce8031b97c27 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:27.742760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f461e-0c84-4aea-b6d0-945f8b4aca21 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:27.780387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf4860-4011-409d-993c-05b45aa5a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:27.780920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf4860-4011-409d-993c-05b45aa5a82b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.781199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf4860-4011-409d-993c-05b45aa5a82b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.781465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf4860-4011-409d-993c-05b45aa5a82b None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.782192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf4860-4011-409d-993c-05b45aa5a82b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:27.782486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf4860-4011-409d-993c-05b45aa5a82b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:27.783851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf4860-4011-409d-993c-05b45aa5a82b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:27.792043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 18/32] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:27.793206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9d88-4c45-4636-8ea6-8a9a7e04d7ad None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.793493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9d88-4c45-4636-8ea6-8a9a7e04d7ad None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.793792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9d88-4c45-4636-8ea6-8a9a7e04d7ad None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.795011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 19/33] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:27.801162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9d88-4c45-4636-8ea6-8a9a7e04d7ad None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:27.804549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:27.898592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9d88-4c45-4636-8ea6-8a9a7e04d7ad None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:27.919153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 15/34] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Aug 23 07:54:27 2022] POST /identity/v3/roles => generated 225 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:27.919153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0657e-84f2-48b6-957f-ce8031b97c27 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:27.940488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:27.945642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:27.946687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:27.947013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:27.947289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:27.947953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:27.948825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:27.950596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:27.978425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 20/35] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:27.979269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:28.071916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0657e-84f2-48b6-957f-ce8031b97c27 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:28.076696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:28.120891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427104a-c2b6-4e8d-aced-1c18a369a1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:28.121524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427104a-c2b6-4e8d-aced-1c18a369a1e2 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.121734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a23fc6-580b-47fa-b91b-bc1ae5e7dd3e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:28.121832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427104a-c2b6-4e8d-aced-1c18a369a1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.122076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427104a-c2b6-4e8d-aced-1c18a369a1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.122752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427104a-c2b6-4e8d-aced-1c18a369a1e2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:28.124887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427104a-c2b6-4e8d-aced-1c18a369a1e2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:28.159114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee78bf-0033-4e98-acd7-e0307ef3422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:28.159745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee78bf-0033-4e98-acd7-e0307ef3422e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.160024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee78bf-0033-4e98-acd7-e0307ef3422e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.160296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee78bf-0033-4e98-acd7-e0307ef3422e None admin] PATH_INFO: `/v3/users/70cd347e5af341fcb03bb3178f2c4247` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.161612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee78bf-0033-4e98-acd7-e0307ef3422e None admin] RBAC: Authorizing `identity:get_user(user_id=70cd347e5af341fcb03bb3178f2c4247)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:28.163219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee78bf-0033-4e98-acd7-e0307ef3422e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:28.164884 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 21/36] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:27 2022] GET /identity/v3/users/70cd347e5af341fcb03bb3178f2c4247 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:28.166040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66d838-bdb0-4edd-9ceb-a62f23852e51 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.166322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66d838-bdb0-4edd-9ceb-a62f23852e51 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.166671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66d838-bdb0-4edd-9ceb-a62f23852e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.189650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0f66d838-bdb0-4edd-9ceb-a62f23852e51 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:28.190844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0f66d838-bdb0-4edd-9ceb-a62f23852e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:28.196564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 16/37] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Aug 23 07:54:27 2022] POST /identity/v3/roles => generated 219 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:28.197642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427104a-c2b6-4e8d-aced-1c18a369a1e2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:28.288918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:28.292360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 22/38] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:28.293758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66d838-bdb0-4edd-9ceb-a62f23852e51 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:28.328757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427104a-c2b6-4e8d-aced-1c18a369a1e2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:28.364491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d554b-527e-48ff-9156-606bea3fc834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:28.365213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d554b-527e-48ff-9156-606bea3fc834 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.365569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d554b-527e-48ff-9156-606bea3fc834 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.365911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d554b-527e-48ff-9156-606bea3fc834 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.366642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d554b-527e-48ff-9156-606bea3fc834 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:28.368320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d554b-527e-48ff-9156-606bea3fc834 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:28.374486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:28.416861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66d838-bdb0-4edd-9ceb-a62f23852e51 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:28.455960 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedde4b-5e56-47a6-aacd-eefee1e55232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:28.456631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedde4b-5e56-47a6-aacd-eefee1e55232 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.456960 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedde4b-5e56-47a6-aacd-eefee1e55232 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.457224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedde4b-5e56-47a6-aacd-eefee1e55232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.458334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedde4b-5e56-47a6-aacd-eefee1e55232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:28.459850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedde4b-5e56-47a6-aacd-eefee1e55232 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:28.461406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 23/39] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:28.462495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba39ed-b094-46e4-b670-cb9b2d337bbf None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.462773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba39ed-b094-46e4-b670-cb9b2d337bbf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.463066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba39ed-b094-46e4-b670-cb9b2d337bbf None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.464553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 24/40] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:28.467725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba39ed-b094-46e4-b670-cb9b2d337bbf None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:28.507025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 17/41] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:54:28 2022] POST /identity/v3/projects => generated 301 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:28.521061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5e1c-0c4e-43e8-b021-a669e10ae654 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.521375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5e1c-0c4e-43e8-b021-a669e10ae654 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.521615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5e1c-0c4e-43e8-b021-a669e10ae654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.551134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-643d5e1c-0c4e-43e8-b021-a669e10ae654 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:28.553710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-643d5e1c-0c4e-43e8-b021-a669e10ae654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:28.593306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba39ed-b094-46e4-b670-cb9b2d337bbf None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:28.634728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:28.640686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3ec0-6e1c-4345-a473-2f5a2302b551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:28.641345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3ec0-6e1c-4345-a473-2f5a2302b551 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.641618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3ec0-6e1c-4345-a473-2f5a2302b551 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.642188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3ec0-6e1c-4345-a473-2f5a2302b551 None admin] PATH_INFO: `/v3/domains/default/users/70cd347e5af341fcb03bb3178f2c4247/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.646069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 18/42] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:28.646069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d5e1c-0c4e-43e8-b021-a669e10ae654 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:28.655247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a3ec0-6e1c-4345-a473-2f5a2302b551 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=70cd347e5af341fcb03bb3178f2c4247, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:28.656723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a3ec0-6e1c-4345-a473-2f5a2302b551 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:28.806786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:28.856106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d5e1c-0c4e-43e8-b021-a669e10ae654 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:28.865344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 25/43] 10.222.0.58 () {58 vars in 1361 bytes} [Tue Aug 23 07:54:28 2022] PUT /identity/v3/domains/default/users/70cd347e5af341fcb03bb3178f2c4247/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:54:28.894179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc658f-d79d-4a2d-b12b-26b7088be380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:28.894809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc658f-d79d-4a2d-b12b-26b7088be380 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:28.895143 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc658f-d79d-4a2d-b12b-26b7088be380 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:28.895416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc658f-d79d-4a2d-b12b-26b7088be380 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:28.896097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc658f-d79d-4a2d-b12b-26b7088be380 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:28.897570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc658f-d79d-4a2d-b12b-26b7088be380 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:28.916679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3ec0-6e1c-4345-a473-2f5a2302b551 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:28.986987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 19/44] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Aug 23 07:54:28 2022] POST /identity/v3/roles => generated 223 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:29.031938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3ec0-6e1c-4345-a473-2f5a2302b551 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:29.072212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b953b5-15d7-4bfd-b038-b55c3478f451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:29.072866 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b953b5-15d7-4bfd-b038-b55c3478f451 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:29.073146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b953b5-15d7-4bfd-b038-b55c3478f451 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:29.073416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b953b5-15d7-4bfd-b038-b55c3478f451 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:29.074666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b953b5-15d7-4bfd-b038-b55c3478f451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:29.076426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b953b5-15d7-4bfd-b038-b55c3478f451 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:29.081186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 26/45] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:29.086684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc658f-d79d-4a2d-b12b-26b7088be380 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:29.205333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc658f-d79d-4a2d-b12b-26b7088be380 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:29.216468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:29.240281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc52f9-83e9-46d2-a4ad-70dfe16fae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:29.240889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc52f9-83e9-46d2-a4ad-70dfe16fae5b None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:29.241170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc52f9-83e9-46d2-a4ad-70dfe16fae5b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:29.241445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc52f9-83e9-46d2-a4ad-70dfe16fae5b None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:29.242083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc52f9-83e9-46d2-a4ad-70dfe16fae5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:29.243657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc52f9-83e9-46d2-a4ad-70dfe16fae5b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:29.372965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 20/46] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:54:29 2022] POST /identity/v3/projects => generated 312 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:30.769420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d05c87-ebe9-411d-937d-9833c2977965 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:30.769952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d05c87-ebe9-411d-937d-9833c2977965 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:30.769952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d05c87-ebe9-411d-937d-9833c2977965 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:30.771476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 27/47] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:30.776870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75d501-a520-4427-b139-4934a4f46a8c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:30.777263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75d501-a520-4427-b139-4934a4f46a8c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:30.777429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75d501-a520-4427-b139-4934a4f46a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:30.795923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ba75d501-a520-4427-b139-4934a4f46a8c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:30.797113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ba75d501-a520-4427-b139-4934a4f46a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:30.862339 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 21/48] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:31.129202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d05c87-ebe9-411d-937d-9833c2977965 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:31.188891 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:31.230233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d05c87-ebe9-411d-937d-9833c2977965 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:31.262917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980942-f5ef-4b09-9f94-7d37311e8b3e None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.263239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980942-f5ef-4b09-9f94-7d37311e8b3e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.264103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980942-f5ef-4b09-9f94-7d37311e8b3e None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.265457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 22/49] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:31.266520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cc68a-9d66-4007-93e1-c87d5f8ff327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:31.267178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cc68a-9d66-4007-93e1-c87d5f8ff327 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.267462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cc68a-9d66-4007-93e1-c87d5f8ff327 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.267738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cc68a-9d66-4007-93e1-c87d5f8ff327 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.275601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe96e4-eab8-4ac5-99ff-cd18edc05850 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.275892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe96e4-eab8-4ac5-99ff-cd18edc05850 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.276169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe96e4-eab8-4ac5-99ff-cd18edc05850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.282832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cc68a-9d66-4007-93e1-c87d5f8ff327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-602cc68a-9d66-4007-93e1-c87d5f8ff327 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:31.291921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:31.292961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:31.293777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:31.294345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bdfe96e4-eab8-4ac5-99ff-cd18edc05850 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:31.295357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bdfe96e4-eab8-4ac5-99ff-cd18edc05850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:31.295459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 28/50] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:54:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:31.306996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cc68a-9d66-4007-93e1-c87d5f8ff327 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:31.361841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 23/51] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:54:31.402861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cc68a-9d66-4007-93e1-c87d5f8ff327 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:31.422500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:31.435770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:31.436366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.450998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.450998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.450998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:31.450998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:31.450998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:31.450998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 29/52] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:31.453341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe96e4-eab8-4ac5-99ff-cd18edc05850 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:31.477769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:31.554080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe96e4-eab8-4ac5-99ff-cd18edc05850 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:31.588892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe68a-8d34-4663-84c5-9e92a1e8a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:31.589522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe68a-8d34-4663-84c5-9e92a1e8a3bf None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.589805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe68a-8d34-4663-84c5-9e92a1e8a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.590081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe68a-8d34-4663-84c5-9e92a1e8a3bf None admin] PATH_INFO: `/v3/users/70cd347e5af341fcb03bb3178f2c4247` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.591259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efe68a-8d34-4663-84c5-9e92a1e8a3bf None admin] RBAC: Authorizing `identity:get_user(user_id=70cd347e5af341fcb03bb3178f2c4247)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:31.593364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efe68a-8d34-4663-84c5-9e92a1e8a3bf None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:31.595086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 24/53] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:31 2022] GET /identity/v3/users/70cd347e5af341fcb03bb3178f2c4247 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:31.600897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:31.623592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe68a-8d34-4663-84c5-9e92a1e8a3bf None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:31.702268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc9de-ca22-4b9b-b6c5-7def8abe396c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:31.715496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:31.726246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe68a-8d34-4663-84c5-9e92a1e8a3bf None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:31.762752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca33d3-4ec7-436d-893b-a2705e8fa6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:31.764008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca33d3-4ec7-436d-893b-a2705e8fa6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.764287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca33d3-4ec7-436d-893b-a2705e8fa6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.764557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca33d3-4ec7-436d-893b-a2705e8fa6e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.765744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca33d3-4ec7-436d-893b-a2705e8fa6e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:31.767265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca33d3-4ec7-436d-893b-a2705e8fa6e4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:31.768884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 25/54] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:31.774747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca33d3-4ec7-436d-893b-a2705e8fa6e4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:31.845100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4275e9-3480-4f2a-a186-b7d8af12fd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:31.845749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4275e9-3480-4f2a-a186-b7d8af12fd82 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.846030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4275e9-3480-4f2a-a186-b7d8af12fd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.846305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4275e9-3480-4f2a-a186-b7d8af12fd82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.847544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4275e9-3480-4f2a-a186-b7d8af12fd82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:31.848982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4275e9-3480-4f2a-a186-b7d8af12fd82 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:31.850563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 30/55] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:31.856357 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4275e9-3480-4f2a-a186-b7d8af12fd82 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:31.878154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca33d3-4ec7-436d-893b-a2705e8fa6e4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:31.907668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:31.912047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f79a7f-ace8-4b56-be4e-8b976e2047ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:31.912657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79a7f-ace8-4b56-be4e-8b976e2047ec None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.912946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79a7f-ace8-4b56-be4e-8b976e2047ec None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.913233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79a7f-ace8-4b56-be4e-8b976e2047ec None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.913875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f79a7f-ace8-4b56-be4e-8b976e2047ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:31.915389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f79a7f-ace8-4b56-be4e-8b976e2047ec None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:31.957167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4275e9-3480-4f2a-a186-b7d8af12fd82 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:31.988891 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb590d0b-e02b-4e14-b992-8fcb8b41fa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:31.989493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb590d0b-e02b-4e14-b992-8fcb8b41fa6f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:31.989779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb590d0b-e02b-4e14-b992-8fcb8b41fa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:31.990059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb590d0b-e02b-4e14-b992-8fcb8b41fa6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:31.990785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb590d0b-e02b-4e14-b992-8fcb8b41fa6f None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.domain.id=default, user.id=70cd347e5af341fcb03bb3178f2c4247) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:31.991129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb590d0b-e02b-4e14-b992-8fcb8b41fa6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:31.992558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb590d0b-e02b-4e14-b992-8fcb8b41fa6f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:32.001450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 26/56] 10.222.0.58 () {60 vars in 1140 bytes} [Tue Aug 23 07:54:31 2022] POST /identity/v3/projects => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:54:32.001679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 31/57] 10.222.0.58 () {56 vars in 1322 bytes} [Tue Aug 23 07:54:31 2022] GET /identity/v3/role_assignments?user.id=70cd347e5af341fcb03bb3178f2c4247&scope.domain.id=default&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:33.890364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c1aeb-e4c3-4305-8416-c6ee1423adc4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:33.890861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c1aeb-e4c3-4305-8416-c6ee1423adc4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:33.890986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c1aeb-e4c3-4305-8416-c6ee1423adc4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:33.892289 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 27/58] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:33.897360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01a81-285c-45d7-aa3c-932709364551 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:33.897774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01a81-285c-45d7-aa3c-932709364551 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:33.897915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01a81-285c-45d7-aa3c-932709364551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:33.916430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f6a01a81-285c-45d7-aa3c-932709364551 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:33.917956 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f6a01a81-285c-45d7-aa3c-932709364551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:33.983224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 32/59] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:34.245550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c1aeb-e4c3-4305-8416-c6ee1423adc4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:34.260574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:34.346618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c1aeb-e4c3-4305-8416-c6ee1423adc4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:34.378521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9dc9c4-a497-4d83-9f09-a488e368a6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:34.379168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dc9c4-a497-4d83-9f09-a488e368a6fb None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:34.379450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dc9c4-a497-4d83-9f09-a488e368a6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:34.379721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9dc9c4-a497-4d83-9f09-a488e368a6fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:34.380836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9dc9c4-a497-4d83-9f09-a488e368a6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:34.382275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9dc9c4-a497-4d83-9f09-a488e368a6fb None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:34.383856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 28/60] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:34.390009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01a81-285c-45d7-aa3c-932709364551 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:34.433291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:34.485965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01a81-285c-45d7-aa3c-932709364551 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:34.517887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5263bc-ff4a-4a4a-bc5f-28c5dce5ed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:34.518431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5263bc-ff4a-4a4a-bc5f-28c5dce5ed2c None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:34.518712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5263bc-ff4a-4a4a-bc5f-28c5dce5ed2c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:34.519011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5263bc-ff4a-4a4a-bc5f-28c5dce5ed2c None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:34.519656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5263bc-ff4a-4a4a-bc5f-28c5dce5ed2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:34.521111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5263bc-ff4a-4a4a-bc5f-28c5dce5ed2c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:34.645315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 33/61] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:54:34 2022] POST /identity/v3/users => generated 275 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:54:37.481238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06450-c174-40fd-ae36-da5483a9b12e None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:37.482295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06450-c174-40fd-ae36-da5483a9b12e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:37.482775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06450-c174-40fd-ae36-da5483a9b12e None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:37.484308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 29/62] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:37.489760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e099e0-9f8e-4399-9433-cd13aa508aaa None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:37.490536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e099e0-9f8e-4399-9433-cd13aa508aaa None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:37.491007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e099e0-9f8e-4399-9433-cd13aa508aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:37.513070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-27e099e0-9f8e-4399-9433-cd13aa508aaa None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:37.515141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-27e099e0-9f8e-4399-9433-cd13aa508aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:37.609719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 34/63] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:37.713372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8f2fc-be71-4c7f-853a-8f9e05c778fe None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:37.713990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8f2fc-be71-4c7f-853a-8f9e05c778fe None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:37.714434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8f2fc-be71-4c7f-853a-8f9e05c778fe None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:37.716285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 30/64] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:37.721578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480711-4b3d-4acc-b680-81df72ed9b82 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:37.721875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480711-4b3d-4acc-b680-81df72ed9b82 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:37.722160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480711-4b3d-4acc-b680-81df72ed9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:37.748237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5a480711-4b3d-4acc-b680-81df72ed9b82 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:37.749458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5a480711-4b3d-4acc-b680-81df72ed9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:37.782381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:37.845744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 35/65] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:37.875985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c8e8-1c10-4bd5-9c2d-039c38669899 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:37.876570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c8e8-1c10-4bd5-9c2d-039c38669899 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:37.877015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c8e8-1c10-4bd5-9c2d-039c38669899 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:37.878522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 31/66] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:37.883954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620fef-3dc3-4a9b-b418-c399a2d5aef5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:37.884249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620fef-3dc3-4a9b-b418-c399a2d5aef5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:37.884529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620fef-3dc3-4a9b-b418-c399a2d5aef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:37.907908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9b620fef-3dc3-4a9b-b418-c399a2d5aef5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:37.909117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9b620fef-3dc3-4a9b-b418-c399a2d5aef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:37.930664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049c8e8-1c10-4bd5-9c2d-039c38669899 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.010989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 36/67] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:38.049838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:38.073552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049c8e8-1c10-4bd5-9c2d-039c38669899 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.108327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f911f-fc44-40c3-a9d6-31163ac12281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.109308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f911f-fc44-40c3-a9d6-31163ac12281 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.109808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f911f-fc44-40c3-a9d6-31163ac12281 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.110282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f911f-fc44-40c3-a9d6-31163ac12281 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.121055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f911f-fc44-40c3-a9d6-31163ac12281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-173f911f-fc44-40c3-a9d6-31163ac12281 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:38.127123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:38.127960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:38.128762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:38.131390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 32/68] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:38.137400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620fef-3dc3-4a9b-b418-c399a2d5aef5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.155122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f911f-fc44-40c3-a9d6-31163ac12281 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.251399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:38.263071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620fef-3dc3-4a9b-b418-c399a2d5aef5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.285365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f911f-fc44-40c3-a9d6-31163ac12281 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.301796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f521f9-96e3-4f3e-920c-fdff02ee2f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.302762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f521f9-96e3-4f3e-920c-fdff02ee2f34 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.303218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f521f9-96e3-4f3e-920c-fdff02ee2f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.303665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f521f9-96e3-4f3e-920c-fdff02ee2f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.304484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f521f9-96e3-4f3e-920c-fdff02ee2f34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:38.304937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f521f9-96e3-4f3e-920c-fdff02ee2f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:38.306642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f521f9-96e3-4f3e-920c-fdff02ee2f34 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:38.316169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 37/69] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:38.317786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaedf31-debc-4904-b715-20aa78488093 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.318272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaedf31-debc-4904-b715-20aa78488093 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.318740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaedf31-debc-4904-b715-20aa78488093 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.320410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 38/70] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:38.325457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a189fb-58be-453e-be6c-95bc5f8fdbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.326633 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a189fb-58be-453e-be6c-95bc5f8fdbad None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.326919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a189fb-58be-453e-be6c-95bc5f8fdbad None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.327037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaedf31-debc-4904-b715-20aa78488093 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.327230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a189fb-58be-453e-be6c-95bc5f8fdbad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.335670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:38.340201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a189fb-58be-453e-be6c-95bc5f8fdbad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-62a189fb-58be-453e-be6c-95bc5f8fdbad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:38.346843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:38.349230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:38.349996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 33/71] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:54:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:38.350667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad509fc6-376e-4efe-a8e6-bbe5a8ec182a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.350730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad509fc6-376e-4efe-a8e6-bbe5a8ec182a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.351227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad509fc6-376e-4efe-a8e6-bbe5a8ec182a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.373946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ad509fc6-376e-4efe-a8e6-bbe5a8ec182a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:38.375724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ad509fc6-376e-4efe-a8e6-bbe5a8ec182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:38.444868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaedf31-debc-4904-b715-20aa78488093 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.451644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 34/72] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:38.452940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad509fc6-376e-4efe-a8e6-bbe5a8ec182a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.492920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac265a-74ab-4338-a7e5-6fed0a1cc41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.494653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac265a-74ab-4338-a7e5-6fed0a1cc41e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.495158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac265a-74ab-4338-a7e5-6fed0a1cc41e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.495615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac265a-74ab-4338-a7e5-6fed0a1cc41e None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.517006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac265a-74ab-4338-a7e5-6fed0a1cc41e None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:38.519381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac265a-74ab-4338-a7e5-6fed0a1cc41e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:38.521433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 39/73] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:38 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:38.522369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac265a-74ab-4338-a7e5-6fed0a1cc41e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.545894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:38.567488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad509fc6-376e-4efe-a8e6-bbe5a8ec182a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.595656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:38.604056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f8b1c-2efe-459c-8a8d-d13045c76847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.605070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8b1c-2efe-459c-8a8d-d13045c76847 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.605361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8b1c-2efe-459c-8a8d-d13045c76847 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.605643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f8b1c-2efe-459c-8a8d-d13045c76847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.617212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f8b1c-2efe-459c-8a8d-d13045c76847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-3b8f8b1c-2efe-459c-8a8d-d13045c76847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:38.623222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:38.624288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:38.625189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 35/74] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:54:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:38.629721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f8b1c-2efe-459c-8a8d-d13045c76847 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.645037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac265a-74ab-4338-a7e5-6fed0a1cc41e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.681172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f134687-30ea-4ecf-b773-fd3e391ad862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.681776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f134687-30ea-4ecf-b773-fd3e391ad862 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.682055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f134687-30ea-4ecf-b773-fd3e391ad862 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.682323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f134687-30ea-4ecf-b773-fd3e391ad862 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.682997 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f134687-30ea-4ecf-b773-fd3e391ad862 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:38.683300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f134687-30ea-4ecf-b773-fd3e391ad862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:38.684675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f134687-30ea-4ecf-b773-fd3e391ad862 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:38.699252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 40/75] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:54:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:38.699252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf14f7-8edb-40cd-bc19-edc0e642eee6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.699252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf14f7-8edb-40cd-bc19-edc0e642eee6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.699252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf14f7-8edb-40cd-bc19-edc0e642eee6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.702511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 41/76] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:38.703568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf14f7-8edb-40cd-bc19-edc0e642eee6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.738875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f8b1c-2efe-459c-8a8d-d13045c76847 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.774043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2742b74-8aaf-459d-9448-c558bab2bc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.774585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742b74-8aaf-459d-9448-c558bab2bc2b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.774865 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742b74-8aaf-459d-9448-c558bab2bc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.775164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742b74-8aaf-459d-9448-c558bab2bc2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.777040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:38.811870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf14f7-8edb-40cd-bc19-edc0e642eee6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.831551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:38.844238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2742b74-8aaf-459d-9448-c558bab2bc2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:38.845106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.845715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.845806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2742b74-8aaf-459d-9448-c558bab2bc2b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:38.845998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.846273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.846987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:38.847299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:38.848690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:38.856692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 42/77] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:38.859040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:38.943369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 36/78] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:38.944859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883c0a4-cfd8-40cf-8ae0-1a2bc33d5518 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.945146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883c0a4-cfd8-40cf-8ae0-1a2bc33d5518 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.945421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883c0a4-cfd8-40cf-8ae0-1a2bc33d5518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.963110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1be1a-9f8e-4edb-a3ff-67d9cfb402d6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:38.963284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a883c0a4-cfd8-40cf-8ae0-1a2bc33d5518 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:38.964369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a883c0a4-cfd8-40cf-8ae0-1a2bc33d5518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:38.996587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc5f0-f013-4cb6-932f-1548f273c57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:38.997209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc5f0-f013-4cb6-932f-1548f273c57b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:38.997491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc5f0-f013-4cb6-932f-1548f273c57b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:38.997793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc5f0-f013-4cb6-932f-1548f273c57b None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:38.998958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dc5f0-f013-4cb6-932f-1548f273c57b None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.000412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dc5f0-f013-4cb6-932f-1548f273c57b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.001999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 43/79] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:38 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:39.002757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc5f0-f013-4cb6-932f-1548f273c57b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.005277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.032274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 37/80] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:39.033356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883c0a4-cfd8-40cf-8ae0-1a2bc33d5518 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.051481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.113846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc5f0-f013-4cb6-932f-1548f273c57b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.140358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883c0a4-cfd8-40cf-8ae0-1a2bc33d5518 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.149795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3b3a5-1cfc-40bc-a026-4be905504ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.150446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3b3a5-1cfc-40bc-a026-4be905504ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.150732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3b3a5-1cfc-40bc-a026-4be905504ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.151040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3b3a5-1cfc-40bc-a026-4be905504ae3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.161450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af3b3a5-1cfc-40bc-a026-4be905504ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9af3b3a5-1cfc-40bc-a026-4be905504ae3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:39.167904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 44/81] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:39.168777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3b3a5-1cfc-40bc-a026-4be905504ae3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.180947 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68723ae5-0635-4fb7-9624-f123943914b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.181583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68723ae5-0635-4fb7-9624-f123943914b1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.181866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68723ae5-0635-4fb7-9624-f123943914b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.182140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68723ae5-0635-4fb7-9624-f123943914b1 None admin] PATH_INFO: `/v3/users/70cd347e5af341fcb03bb3178f2c4247` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.183689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68723ae5-0635-4fb7-9624-f123943914b1 None admin] RBAC: Authorizing `identity:get_user(user_id=70cd347e5af341fcb03bb3178f2c4247)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.185040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68723ae5-0635-4fb7-9624-f123943914b1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.186872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 38/82] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/users/70cd347e5af341fcb03bb3178f2c4247 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:39.191963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68723ae5-0635-4fb7-9624-f123943914b1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.239034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.279944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3b3a5-1cfc-40bc-a026-4be905504ae3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.281534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.298831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68723ae5-0635-4fb7-9624-f123943914b1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.316616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9e26-fb68-4645-ad04-74b5826154d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.317254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9e26-fb68-4645-ad04-74b5826154d7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.317533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9e26-fb68-4645-ad04-74b5826154d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.317805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9e26-fb68-4645-ad04-74b5826154d7 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.318968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b9e26-fb68-4645-ad04-74b5826154d7 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.320439 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b9e26-fb68-4645-ad04-74b5826154d7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.322348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 45/83] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:39.323129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9e26-fb68-4645-ad04-74b5826154d7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.334919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb1c06-d213-400c-9d6a-d3a9933cc751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.335561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1c06-d213-400c-9d6a-d3a9933cc751 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.335845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1c06-d213-400c-9d6a-d3a9933cc751 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.336137 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1c06-d213-400c-9d6a-d3a9933cc751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.404286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb1c06-d213-400c-9d6a-d3a9933cc751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.405761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb1c06-d213-400c-9d6a-d3a9933cc751 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.422056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9e26-fb68-4645-ad04-74b5826154d7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.449227 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.454713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.455339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.455618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.455896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.456606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:39.456852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.458226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.467040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 46/84] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:39.467820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.491626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.507324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 39/85] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:39.507464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb1c06-d213-400c-9d6a-d3a9933cc751 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.567014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a72045-c837-4151-b957-fe3bf9a4651d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.604767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb1c06-d213-400c-9d6a-d3a9933cc751 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.605250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f72e-59f7-49cf-99f4-01e0f40755fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.605737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f72e-59f7-49cf-99f4-01e0f40755fc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.606012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f72e-59f7-49cf-99f4-01e0f40755fc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.606280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229f72e-59f7-49cf-99f4-01e0f40755fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.638398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd29c49-fbf6-4b3f-9305-19fffd0a336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.639073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29c49-fbf6-4b3f-9305-19fffd0a336f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.639348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29c49-fbf6-4b3f-9305-19fffd0a336f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.639625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29c49-fbf6-4b3f-9305-19fffd0a336f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.640720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd29c49-fbf6-4b3f-9305-19fffd0a336f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.642164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd29c49-fbf6-4b3f-9305-19fffd0a336f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.643737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 40/86] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:39.644601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd29c49-fbf6-4b3f-9305-19fffd0a336f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.664433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.674644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229f72e-59f7-49cf-99f4-01e0f40755fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.676188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229f72e-59f7-49cf-99f4-01e0f40755fc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.698747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.742119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd29c49-fbf6-4b3f-9305-19fffd0a336f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.773679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 47/87] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:39.774406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f72e-59f7-49cf-99f4-01e0f40755fc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.782224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.782839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.783149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.783423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.784159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] RBAC: Adding query filter params (role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=772dfdcbe95541519b7335820a928256, user.id=78837c5f7b204c9c96f712838f987797) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:39.784471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.785951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.791500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 41/88] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/role_assignments?user.id=78837c5f7b204c9c96f712838f987797&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=3b325dcac3064617a5415a84759ef71a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:39.792282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.878157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.888525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229f72e-59f7-49cf-99f4-01e0f40755fc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.924149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba57a-b828-4ee5-a480-f2b14236b558 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:39.931558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ae4d4-f15b-4a67-a51a-c85141e31918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.932637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ae4d4-f15b-4a67-a51a-c85141e31918 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.933348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ae4d4-f15b-4a67-a51a-c85141e31918 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.933635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ae4d4-f15b-4a67-a51a-c85141e31918 None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.935311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ae4d4-f15b-4a67-a51a-c85141e31918 None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.937093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ae4d4-f15b-4a67-a51a-c85141e31918 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.938987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 48/89] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:39.939739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ae4d4-f15b-4a67-a51a-c85141e31918 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:39.947703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:39.964949 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b353c2-b9c8-4545-a75d-79dc8ec6fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:39.965649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b353c2-b9c8-4545-a75d-79dc8ec6fcec None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:39.965932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b353c2-b9c8-4545-a75d-79dc8ec6fcec None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:39.966215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b353c2-b9c8-4545-a75d-79dc8ec6fcec None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:39.968048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b353c2-b9c8-4545-a75d-79dc8ec6fcec None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:39.969498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b353c2-b9c8-4545-a75d-79dc8ec6fcec None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:39.971154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 42/90] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:39.971915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b353c2-b9c8-4545-a75d-79dc8ec6fcec None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:40.052946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ae4d4-f15b-4a67-a51a-c85141e31918 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:40.083046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b353c2-b9c8-4545-a75d-79dc8ec6fcec None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:40.090045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4701d-873c-44cd-a4e1-d5d6936a775d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:40.090654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4701d-873c-44cd-a4e1-d5d6936a775d None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:40.090920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4701d-873c-44cd-a4e1-d5d6936a775d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:40.091215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4701d-873c-44cd-a4e1-d5d6936a775d None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:40.091913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4701d-873c-44cd-a4e1-d5d6936a775d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:40.093358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4701d-873c-44cd-a4e1-d5d6936a775d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:40.118493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89bd89-d6d9-43be-905e-7f6bb6a32ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:40.123375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bd89-d6d9-43be-905e-7f6bb6a32ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:40.123375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bd89-d6d9-43be-905e-7f6bb6a32ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:40.123375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89bd89-d6d9-43be-905e-7f6bb6a32ed9 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:40.123375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89bd89-d6d9-43be-905e-7f6bb6a32ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:40.123375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89bd89-d6d9-43be-905e-7f6bb6a32ed9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:40.124075 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 43/91] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:39 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:40.128947 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89bd89-d6d9-43be-905e-7f6bb6a32ed9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:40.181077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:40.202980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 49/92] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:54:39 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:40.202980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4701d-873c-44cd-a4e1-d5d6936a775d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:40.229873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:40.247115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89bd89-d6d9-43be-905e-7f6bb6a32ed9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:40.286726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a19c5b-6fa4-41a7-b633-fc31b2314165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:40.287941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a19c5b-6fa4-41a7-b633-fc31b2314165 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:40.288356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a19c5b-6fa4-41a7-b633-fc31b2314165 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:40.289067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a19c5b-6fa4-41a7-b633-fc31b2314165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:40.348548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4701d-873c-44cd-a4e1-d5d6936a775d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:40.382868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a19c5b-6fa4-41a7-b633-fc31b2314165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:40.384311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a19c5b-6fa4-41a7-b633-fc31b2314165 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:40.390841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:40.391514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:40.392376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:40.392376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:40.393177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] RBAC: Adding query filter params (role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=772dfdcbe95541519b7335820a928256, user.id=78837c5f7b204c9c96f712838f987797) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:40.393277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:40.394697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:40.405558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 50/93] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:40 2022] GET /identity/v3/role_assignments?user.id=78837c5f7b204c9c96f712838f987797&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:40.407566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:40.461892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:40.521015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 44/94] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:40.527192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a19c5b-6fa4-41a7-b633-fc31b2314165 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:40.530491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:40.551440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b29ed-32a4-42e3-93d7-a488481c19a3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:40.591730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:40.592488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:40.592865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:40.593236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:40.594067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=772dfdcbe95541519b7335820a928256, user.id=70cd347e5af341fcb03bb3178f2c4247) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:40.594477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:40.596061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:40.602368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 51/95] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:40 2022] GET /identity/v3/role_assignments?user.id=70cd347e5af341fcb03bb3178f2c4247&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:40.646745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a19c5b-6fa4-41a7-b633-fc31b2314165 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:40.687573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5185f9-939f-4101-86d2-c4f575a177f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:40.688463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5185f9-939f-4101-86d2-c4f575a177f1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:40.688887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5185f9-939f-4101-86d2-c4f575a177f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:40.689303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5185f9-939f-4101-86d2-c4f575a177f1 None admin] PATH_INFO: `/v3/projects/5721358b3bfa4bc9898e7b80d7fb6299` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:40.690733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5185f9-939f-4101-86d2-c4f575a177f1 None admin] RBAC: Authorizing `identity:get_project(project_id=5721358b3bfa4bc9898e7b80d7fb6299)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:40.692479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5185f9-939f-4101-86d2-c4f575a177f1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:40.694364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 45/96] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:40 2022] GET /identity/v3/projects/5721358b3bfa4bc9898e7b80d7fb6299 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:40.700871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:40.831798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51b56-5cb4-4893-b7fe-9475aaa40be2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:40.868254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fce13-4c9b-4bdf-86b0-fbab8f2a0549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:40.868867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fce13-4c9b-4bdf-86b0-fbab8f2a0549 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:40.869150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fce13-4c9b-4bdf-86b0-fbab8f2a0549 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:40.869428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fce13-4c9b-4bdf-86b0-fbab8f2a0549 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:40.870156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fce13-4c9b-4bdf-86b0-fbab8f2a0549 None admin] RBAC: Adding query filter params (role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=5721358b3bfa4bc9898e7b80d7fb6299, user.id=78837c5f7b204c9c96f712838f987797) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:40.870464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fce13-4c9b-4bdf-86b0-fbab8f2a0549 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:40.872003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fce13-4c9b-4bdf-86b0-fbab8f2a0549 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:40.878365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 52/97] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:40 2022] GET /identity/v3/role_assignments?user.id=78837c5f7b204c9c96f712838f987797&scope.project.id=5721358b3bfa4bc9898e7b80d7fb6299&role.id=3b325dcac3064617a5415a84759ef71a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:42.215444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8e3ad-bbad-44cd-a0ea-377c621fcca7 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:42.215979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8e3ad-bbad-44cd-a0ea-377c621fcca7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:42.216053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8e3ad-bbad-44cd-a0ea-377c621fcca7 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:42.217469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 46/98] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:42.222966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3264b5f-41b8-4b2b-a693-8bce489bc314 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:42.223384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3264b5f-41b8-4b2b-a693-8bce489bc314 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:42.223572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3264b5f-41b8-4b2b-a693-8bce489bc314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:42.247247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c3264b5f-41b8-4b2b-a693-8bce489bc314 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:42.249023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c3264b5f-41b8-4b2b-a693-8bce489bc314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:42.251104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:42.336590 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 53/99] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:42.708650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1385ea-5d78-490a-b637-a7d8ed2095e3 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:42.709042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1385ea-5d78-490a-b637-a7d8ed2095e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:42.709217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1385ea-5d78-490a-b637-a7d8ed2095e3 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:42.710565 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 47/100] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:42.724397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06d852-bb42-4318-b2c7-b9accc0c4a3a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:42.724691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06d852-bb42-4318-b2c7-b9accc0c4a3a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:42.724975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06d852-bb42-4318-b2c7-b9accc0c4a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:42.751934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1c06d852-bb42-4318-b2c7-b9accc0c4a3a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:42.753151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1c06d852-bb42-4318-b2c7-b9accc0c4a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:42.844583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 54/101] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:42.868364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1385ea-5d78-490a-b637-a7d8ed2095e3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:42.898416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:42.992705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1385ea-5d78-490a-b637-a7d8ed2095e3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:43.001915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf83819-507a-432a-ac49-e7d23c3a21e4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.002206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf83819-507a-432a-ac49-e7d23c3a21e4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.002487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf83819-507a-432a-ac49-e7d23c3a21e4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.003909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 55/102] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:43.009020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e29bc8-c8e8-46fa-8d80-5089b1abf4a5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.009310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e29bc8-c8e8-46fa-8d80-5089b1abf4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.009589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e29bc8-c8e8-46fa-8d80-5089b1abf4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.038017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-08e29bc8-c8e8-46fa-8d80-5089b1abf4a5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:43.039250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-08e29bc8-c8e8-46fa-8d80-5089b1abf4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:43.059805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34854b67-3eae-4626-86c9-1fc195c6483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:43.060551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34854b67-3eae-4626-86c9-1fc195c6483f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.060837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34854b67-3eae-4626-86c9-1fc195c6483f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.061145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34854b67-3eae-4626-86c9-1fc195c6483f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.080630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34854b67-3eae-4626-86c9-1fc195c6483f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-34854b67-3eae-4626-86c9-1fc195c6483f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:43.090251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:43.091112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:43.091948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 48/103] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:43.097216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34854b67-3eae-4626-86c9-1fc195c6483f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:43.151300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:43.157384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 56/104] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:54:43.227108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:43.245405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34854b67-3eae-4626-86c9-1fc195c6483f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:43.248482 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e29bc8-c8e8-46fa-8d80-5089b1abf4a5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:43.281967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:43.282734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.283049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.283326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.284000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:43.284307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:43.285705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:43.299382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 49/105] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:43.308507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:43.460762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e29bc8-c8e8-46fa-8d80-5089b1abf4a5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:43.512729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a61af4-2ed9-4841-b180-1629ef36cb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:43.513814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a61af4-2ed9-4841-b180-1629ef36cb58 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.514090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a61af4-2ed9-4841-b180-1629ef36cb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.514356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a61af4-2ed9-4841-b180-1629ef36cb58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.526169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88324e82-26e5-4ca4-9f34-f023c1b49935 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:43.540207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a61af4-2ed9-4841-b180-1629ef36cb58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-51a61af4-2ed9-4841-b180-1629ef36cb58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:43.545526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:43.546367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:43.547228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 57/106] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:54:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:43.555853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a61af4-2ed9-4841-b180-1629ef36cb58 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:43.587363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d52126-312e-4fb2-85aa-233bfd7625e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:43.588376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d52126-312e-4fb2-85aa-233bfd7625e7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.588849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d52126-312e-4fb2-85aa-233bfd7625e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.589366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d52126-312e-4fb2-85aa-233bfd7625e7 None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.591956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d52126-312e-4fb2-85aa-233bfd7625e7 None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:43.595079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d52126-312e-4fb2-85aa-233bfd7625e7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:43.597096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 50/107] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:43 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:43.602568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d52126-312e-4fb2-85aa-233bfd7625e7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:43.628303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:43.664029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:43.710830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a61af4-2ed9-4841-b180-1629ef36cb58 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:43.753846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:43.759801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d52126-312e-4fb2-85aa-233bfd7625e7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:43.760404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.760956 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.761444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.762384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:43.763206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:43.764805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:43.773661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 58/108] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:54:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:43.774738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:43.799453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a3a7c-8198-4d33-9c61-e28a4e212551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:43.800085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a3a7c-8198-4d33-9c61-e28a4e212551 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.800363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a3a7c-8198-4d33-9c61-e28a4e212551 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.800637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a3a7c-8198-4d33-9c61-e28a4e212551 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.802148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a3a7c-8198-4d33-9c61-e28a4e212551 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:43.803645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a3a7c-8198-4d33-9c61-e28a4e212551 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:43.805554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 51/109] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:43 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:43.806555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d930bb1-76b5-4fa5-bb6a-4968d8b84d18 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.806836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d930bb1-76b5-4fa5-bb6a-4968d8b84d18 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.807134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d930bb1-76b5-4fa5-bb6a-4968d8b84d18 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.808329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 52/110] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:43.809024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d930bb1-76b5-4fa5-bb6a-4968d8b84d18 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:43.895393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:43.896917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c0c6e-2df9-4cf5-a1fb-2520fe9d18e2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:43.916932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d930bb1-76b5-4fa5-bb6a-4968d8b84d18 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:43.918088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:43.933422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534e344-442c-4cd2-a472-f4b3d949cda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:43.934289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e344-442c-4cd2-a472-f4b3d949cda9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.934751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e344-442c-4cd2-a472-f4b3d949cda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.935373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e344-442c-4cd2-a472-f4b3d949cda9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.947283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534e344-442c-4cd2-a472-f4b3d949cda9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f534e344-442c-4cd2-a472-f4b3d949cda9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:43.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:43.953352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:54:43.954100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:43.954699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255665c4-f9ee-4b27-8b94-6309a1a3b03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:43.954783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 59/111] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:43.954856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255665c4-f9ee-4b27-8b94-6309a1a3b03d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.954856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255665c4-f9ee-4b27-8b94-6309a1a3b03d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.954856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255665c4-f9ee-4b27-8b94-6309a1a3b03d None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.956272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255665c4-f9ee-4b27-8b94-6309a1a3b03d None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:43.957716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255665c4-f9ee-4b27-8b94-6309a1a3b03d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:43.959453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 53/112] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:43 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:43.959565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bced91-ec30-4d55-9893-355889cd9dff None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.959786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bced91-ec30-4d55-9893-355889cd9dff None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.960060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bced91-ec30-4d55-9893-355889cd9dff None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.961685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 60/113] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:43.962589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bced91-ec30-4d55-9893-355889cd9dff None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:43.969378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2aa9c-f5ba-4717-b36f-e1597552174f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:43.969679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2aa9c-f5ba-4717-b36f-e1597552174f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:43.969960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2aa9c-f5ba-4717-b36f-e1597552174f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:43.988183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6ec2aa9c-f5ba-4717-b36f-e1597552174f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:43.989436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6ec2aa9c-f5ba-4717-b36f-e1597552174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:44.056894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 54/114] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:44.058062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2aa9c-f5ba-4717-b36f-e1597552174f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:44.073459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bced91-ec30-4d55-9893-355889cd9dff None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:44.109134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc99750-bb67-4734-b122-0ef367d9d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:44.109800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc99750-bb67-4734-b122-0ef367d9d45b None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.110079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc99750-bb67-4734-b122-0ef367d9d45b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.110350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc99750-bb67-4734-b122-0ef367d9d45b None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256/users/78837c5f7b204c9c96f712838f987797/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.112249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc99750-bb67-4734-b122-0ef367d9d45b None admin] RBAC: Authorizing `identity:create_grant(project_id=772dfdcbe95541519b7335820a928256, user_id=78837c5f7b204c9c96f712838f987797, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:44.113679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc99750-bb67-4734-b122-0ef367d9d45b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:44.142521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:44.163591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2aa9c-f5ba-4717-b36f-e1597552174f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:44.197713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c616d4-62c0-41be-9ff6-16e60d5c59d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:44.198343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c616d4-62c0-41be-9ff6-16e60d5c59d7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.198627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c616d4-62c0-41be-9ff6-16e60d5c59d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.198907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c616d4-62c0-41be-9ff6-16e60d5c59d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.199593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c616d4-62c0-41be-9ff6-16e60d5c59d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:44.199928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c616d4-62c0-41be-9ff6-16e60d5c59d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:44.211286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 61/115] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:54:43 2022] PUT /identity/v3/projects/772dfdcbe95541519b7335820a928256/users/78837c5f7b204c9c96f712838f987797/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:54:44.211286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc99750-bb67-4734-b122-0ef367d9d45b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:44.211556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c616d4-62c0-41be-9ff6-16e60d5c59d7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:44.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 55/116] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:44.222695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe71930-e4dd-49af-acef-781dcea7aec3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.223014 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe71930-e4dd-49af-acef-781dcea7aec3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.223287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe71930-e4dd-49af-acef-781dcea7aec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.245933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ebe71930-e4dd-49af-acef-781dcea7aec3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:44.249862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ebe71930-e4dd-49af-acef-781dcea7aec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:44.264748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:44.333471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc99750-bb67-4734-b122-0ef367d9d45b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:44.345441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 56/117] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:44.346885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b4e5-b6d5-469c-9430-5c43fdccf9af None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.347204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b4e5-b6d5-469c-9430-5c43fdccf9af None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.347479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b4e5-b6d5-469c-9430-5c43fdccf9af None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.348868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 57/118] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:44.353574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b4e5-b6d5-469c-9430-5c43fdccf9af None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:44.379726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba138d-a7eb-4d3b-8f6b-30de908bc629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:44.381072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba138d-a7eb-4d3b-8f6b-30de908bc629 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.381562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba138d-a7eb-4d3b-8f6b-30de908bc629 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.382208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba138d-a7eb-4d3b-8f6b-30de908bc629 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.383919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba138d-a7eb-4d3b-8f6b-30de908bc629 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:44.385663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba138d-a7eb-4d3b-8f6b-30de908bc629 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:44.387580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 62/119] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:44 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:44.389060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba138d-a7eb-4d3b-8f6b-30de908bc629 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:44.437829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:44.473254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b4e5-b6d5-469c-9430-5c43fdccf9af None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:44.513639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c0425-eeed-42cd-90b2-bfa8c9991a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:44.514589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c0425-eeed-42cd-90b2-bfa8c9991a7e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.515092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c0425-eeed-42cd-90b2-bfa8c9991a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.515538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c0425-eeed-42cd-90b2-bfa8c9991a7e None admin] PATH_INFO: `/v3/users/6e1fabf461ea4de49e12baeb607e84d4` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.532256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba138d-a7eb-4d3b-8f6b-30de908bc629 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:44.535912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c0425-eeed-42cd-90b2-bfa8c9991a7e None admin] RBAC: Authorizing `identity:get_user(user_id=6e1fabf461ea4de49e12baeb607e84d4)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:44.537475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c0425-eeed-42cd-90b2-bfa8c9991a7e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:44.539677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 58/120] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:44 2022] GET /identity/v3/users/6e1fabf461ea4de49e12baeb607e84d4 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:44.541497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6416b-c8d4-43db-8816-5ed86d6b6ad6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.541784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6416b-c8d4-43db-8816-5ed86d6b6ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.547240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6416b-c8d4-43db-8816-5ed86d6b6ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.547354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:44.565573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-51d6416b-c8d4-43db-8816-5ed86d6b6ad6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:44.567649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-51d6416b-c8d4-43db-8816-5ed86d6b6ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:44.572504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb69dc5-1c8e-43b9-9f78-f6eec70a9b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:44.573426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69dc5-1c8e-43b9-9f78-f6eec70a9b62 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.573876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69dc5-1c8e-43b9-9f78-f6eec70a9b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.574377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69dc5-1c8e-43b9-9f78-f6eec70a9b62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.576288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb69dc5-1c8e-43b9-9f78-f6eec70a9b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:44.577960 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb69dc5-1c8e-43b9-9f78-f6eec70a9b62 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:44.579709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 63/121] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:44.582507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb69dc5-1c8e-43b9-9f78-f6eec70a9b62 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:44.691626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 59/122] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:44.692714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6416b-c8d4-43db-8816-5ed86d6b6ad6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:44.757943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb69dc5-1c8e-43b9-9f78-f6eec70a9b62 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:44.763260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:44.806340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10007480-13a3-4b15-ac7b-734226cd1181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:44.807056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10007480-13a3-4b15-ac7b-734226cd1181 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.807344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10007480-13a3-4b15-ac7b-734226cd1181 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.807621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10007480-13a3-4b15-ac7b-734226cd1181 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256/users/78837c5f7b204c9c96f712838f987797/roles/af615e7795ae42019a45dfe2bb205c49` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.809988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10007480-13a3-4b15-ac7b-734226cd1181 None admin] RBAC: Authorizing `identity:create_grant(project_id=772dfdcbe95541519b7335820a928256, user_id=78837c5f7b204c9c96f712838f987797, role_id=af615e7795ae42019a45dfe2bb205c49)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:44.811623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10007480-13a3-4b15-ac7b-734226cd1181 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:44.815436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6416b-c8d4-43db-8816-5ed86d6b6ad6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:44.852159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5becd-2547-4061-b840-ef2db6528bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:44.852771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5becd-2547-4061-b840-ef2db6528bcc None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:44.853053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5becd-2547-4061-b840-ef2db6528bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:44.853326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5becd-2547-4061-b840-ef2db6528bcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:44.883364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 64/123] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:54:44 2022] PUT /identity/v3/projects/772dfdcbe95541519b7335820a928256/users/78837c5f7b204c9c96f712838f987797/roles/af615e7795ae42019a45dfe2bb205c49 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:54:44.884481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10007480-13a3-4b15-ac7b-734226cd1181 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:44.929338 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:44.948562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5becd-2547-4061-b840-ef2db6528bcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:44.949993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5becd-2547-4061-b840-ef2db6528bcc None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:45.010842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10007480-13a3-4b15-ac7b-734226cd1181 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.016412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:45.054816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03370c43-5374-4d8e-887d-ba01238f9242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:45.055937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03370c43-5374-4d8e-887d-ba01238f9242 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:45.056477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03370c43-5374-4d8e-887d-ba01238f9242 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:45.056982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03370c43-5374-4d8e-887d-ba01238f9242 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:45.057929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03370c43-5374-4d8e-887d-ba01238f9242 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:45.059946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03370c43-5374-4d8e-887d-ba01238f9242 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:45.071510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 60/124] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:45.071510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5becd-2547-4061-b840-ef2db6528bcc None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.147095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 65/125] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:54:44 2022] POST /identity/v3/projects => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:45.148323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03370c43-5374-4d8e-887d-ba01238f9242 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.189743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5becd-2547-4061-b840-ef2db6528bcc None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.229380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598031d9-72ae-4fc3-9fee-0eabcc194fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:45.229812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598031d9-72ae-4fc3-9fee-0eabcc194fea None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:45.230096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598031d9-72ae-4fc3-9fee-0eabcc194fea None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:45.230425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598031d9-72ae-4fc3-9fee-0eabcc194fea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:45.243023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598031d9-72ae-4fc3-9fee-0eabcc194fea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-598031d9-72ae-4fc3-9fee-0eabcc194fea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:45.248065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:45.249332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 61/126] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:45.250062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598031d9-72ae-4fc3-9fee-0eabcc194fea None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.272092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:45.272587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:45.293620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03370c43-5374-4d8e-887d-ba01238f9242 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.339899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a4e0a-264c-476e-aa39-af456a1d11dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:45.342026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a4e0a-264c-476e-aa39-af456a1d11dd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:45.342535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a4e0a-264c-476e-aa39-af456a1d11dd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:45.343120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a4e0a-264c-476e-aa39-af456a1d11dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:45.358374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a4e0a-264c-476e-aa39-af456a1d11dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-652a4e0a-264c-476e-aa39-af456a1d11dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:45.367130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:45.368051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:45.368808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:45.371056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 66/127] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:45.372103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a4e0a-264c-476e-aa39-af456a1d11dd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.372212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598031d9-72ae-4fc3-9fee-0eabcc194fea None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.416028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc8ab5-367d-411e-9462-765a40f6169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:45.416666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc8ab5-367d-411e-9462-765a40f6169d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:45.417000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc8ab5-367d-411e-9462-765a40f6169d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:45.417287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc8ab5-367d-411e-9462-765a40f6169d None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:45.418537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc8ab5-367d-411e-9462-765a40f6169d None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:45.420134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc8ab5-367d-411e-9462-765a40f6169d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:45.422142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 62/128] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:45.422918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc8ab5-367d-411e-9462-765a40f6169d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.511975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a4e0a-264c-476e-aa39-af456a1d11dd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.524699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:45.542042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc8ab5-367d-411e-9462-765a40f6169d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.555003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:45.556067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:45.556515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:45.556932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:45.558231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:45.558688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:45.561067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:45.572235 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 67/129] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:45.573933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.581517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:45.582144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:45.582428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:45.582706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:45.584491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:45.584803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:45.586276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:45.589647 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:45.598246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 63/130] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:45.598951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.704046 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e4ffd-8057-4780-9ece-a29055d9fde0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.713550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6ee0d-ef9a-47a0-9fb4-51c1eefc6b68 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.748927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:45.749570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:45.749841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26506cb-3012-4606-b82a-c1b0eb37c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:45.749953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:45.750115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:45.750891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26506cb-3012-4606-b82a-c1b0eb37c924 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:45.751029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] RBAC: Adding query filter params (role.id=976d0bc0f2b14f26877bcc529d3645c7, scope.project.id=772dfdcbe95541519b7335820a928256, user.id=6e1fabf461ea4de49e12baeb607e84d4) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:45.751287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:45.751688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26506cb-3012-4606-b82a-c1b0eb37c924 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:45.752146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26506cb-3012-4606-b82a-c1b0eb37c924 None admin] PATH_INFO: `/v3/users/70cd347e5af341fcb03bb3178f2c4247` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:45.752757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:45.753692 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26506cb-3012-4606-b82a-c1b0eb37c924 None admin] RBAC: Authorizing `identity:get_user(user_id=70cd347e5af341fcb03bb3178f2c4247)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:45.755473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26506cb-3012-4606-b82a-c1b0eb37c924 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:45.757347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 64/131] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/users/70cd347e5af341fcb03bb3178f2c4247 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:45.758462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26506cb-3012-4606-b82a-c1b0eb37c924 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.758933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 68/132] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/role_assignments?user.id=6e1fabf461ea4de49e12baeb607e84d4&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=976d0bc0f2b14f26877bcc529d3645c7 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:45.764296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:45.812214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:45.940861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:45.984528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26506cb-3012-4606-b82a-c1b0eb37c924 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:45.989639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13dfd7b-a0e9-499d-ae59-47423c4fb9ce None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:46.039686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e011-3e0a-42be-a034-aef64839813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:46.040299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e011-3e0a-42be-a034-aef64839813c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:46.040583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e011-3e0a-42be-a034-aef64839813c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:46.040860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e011-3e0a-42be-a034-aef64839813c None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:46.042119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e011-3e0a-42be-a034-aef64839813c None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:46.043757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e011-3e0a-42be-a034-aef64839813c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:46.045976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 65/133] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:46.051567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e011-3e0a-42be-a034-aef64839813c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:46.067734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a65f5-c817-4459-b19d-3c4003844892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:46.068699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a65f5-c817-4459-b19d-3c4003844892 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:46.069247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a65f5-c817-4459-b19d-3c4003844892 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:46.069717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a65f5-c817-4459-b19d-3c4003844892 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:46.071117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a65f5-c817-4459-b19d-3c4003844892 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:46.073009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a65f5-c817-4459-b19d-3c4003844892 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:46.074914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 69/134] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:45 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:46.080742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a65f5-c817-4459-b19d-3c4003844892 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:46.146701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:46.164468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e011-3e0a-42be-a034-aef64839813c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:46.202330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495fe8b-e0df-4f7f-952f-aa074cd133c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:46.202985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495fe8b-e0df-4f7f-952f-aa074cd133c1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:46.203273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495fe8b-e0df-4f7f-952f-aa074cd133c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:46.203546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495fe8b-e0df-4f7f-952f-aa074cd133c1 None admin] PATH_INFO: `/v3/projects/5721358b3bfa4bc9898e7b80d7fb6299` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:46.204774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495fe8b-e0df-4f7f-952f-aa074cd133c1 None admin] RBAC: Authorizing `identity:get_project(project_id=5721358b3bfa4bc9898e7b80d7fb6299)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:46.206241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495fe8b-e0df-4f7f-952f-aa074cd133c1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:46.208126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 66/135] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:46 2022] GET /identity/v3/projects/5721358b3bfa4bc9898e7b80d7fb6299 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:46.214124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495fe8b-e0df-4f7f-952f-aa074cd133c1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:46.281139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a65f5-c817-4459-b19d-3c4003844892 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:46.323026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340b041-a1cc-4933-97ed-73dbffc054f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:46.324554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340b041-a1cc-4933-97ed-73dbffc054f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:46.325030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340b041-a1cc-4933-97ed-73dbffc054f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:46.325540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340b041-a1cc-4933-97ed-73dbffc054f9 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256/users/70cd347e5af341fcb03bb3178f2c4247/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:46.329610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340b041-a1cc-4933-97ed-73dbffc054f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=772dfdcbe95541519b7335820a928256, user_id=70cd347e5af341fcb03bb3178f2c4247, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:46.331502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340b041-a1cc-4933-97ed-73dbffc054f9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:46.346538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495fe8b-e0df-4f7f-952f-aa074cd133c1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:46.382627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba4539-74d9-4d93-9820-bdec5e48bf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:46.383666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba4539-74d9-4d93-9820-bdec5e48bf2b None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:46.384103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba4539-74d9-4d93-9820-bdec5e48bf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:46.384532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba4539-74d9-4d93-9820-bdec5e48bf2b None admin] PATH_INFO: `/v3/projects/5721358b3bfa4bc9898e7b80d7fb6299/users/78837c5f7b204c9c96f712838f987797/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:46.386630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba4539-74d9-4d93-9820-bdec5e48bf2b None admin] RBAC: Authorizing `identity:create_grant(project_id=5721358b3bfa4bc9898e7b80d7fb6299, user_id=78837c5f7b204c9c96f712838f987797, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:46.388303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba4539-74d9-4d93-9820-bdec5e48bf2b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:46.424252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 70/136] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:54:46 2022] PUT /identity/v3/projects/772dfdcbe95541519b7335820a928256/users/70cd347e5af341fcb03bb3178f2c4247/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:54:46.426223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6397147-7f99-43a7-9900-a137e4f4d4dd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:46.426681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6397147-7f99-43a7-9900-a137e4f4d4dd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:46.427181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6397147-7f99-43a7-9900-a137e4f4d4dd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:46.428922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 71/137] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:46.438021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97beb74-f7bc-4a44-b891-58178fad7b68 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:46.438563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97beb74-f7bc-4a44-b891-58178fad7b68 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:46.439030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97beb74-f7bc-4a44-b891-58178fad7b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:46.446791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:46.463901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c97beb74-f7bc-4a44-b891-58178fad7b68 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:46.469468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c97beb74-f7bc-4a44-b891-58178fad7b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:46.494601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 67/138] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:54:46 2022] PUT /identity/v3/projects/5721358b3bfa4bc9898e7b80d7fb6299/users/78837c5f7b204c9c96f712838f987797/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:54:46.565906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 72/139] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:46.893003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba4539-74d9-4d93-9820-bdec5e48bf2b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:46.914965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:47.026097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba4539-74d9-4d93-9820-bdec5e48bf2b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:47.080868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d26d1-d06b-4e4b-b719-595d2a740739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:47.081847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d26d1-d06b-4e4b-b719-595d2a740739 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:47.082328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d26d1-d06b-4e4b-b719-595d2a740739 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:47.082799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d26d1-d06b-4e4b-b719-595d2a740739 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:47.097993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d26d1-d06b-4e4b-b719-595d2a740739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d41d26d1-d06b-4e4b-b719-595d2a740739 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:47.103510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:47.104341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:47.105099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:47.107341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 68/140] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:47.114681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97beb74-f7bc-4a44-b891-58178fad7b68 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:47.263665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97beb74-f7bc-4a44-b891-58178fad7b68 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:47.308420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:47.313935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:47.314585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:47.314885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:47.332374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:47.332374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:47.332374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:47.332374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:47.332374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 73/141] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:47.336188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d26d1-d06b-4e4b-b719-595d2a740739 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:47.426496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:47.458014 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d26d1-d06b-4e4b-b719-595d2a740739 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:47.502778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b9244-3c4f-41c6-b242-55149bf826b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:47.503796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b9244-3c4f-41c6-b242-55149bf826b8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:47.504277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b9244-3c4f-41c6-b242-55149bf826b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:47.504776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b9244-3c4f-41c6-b242-55149bf826b8 None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:47.506322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b9244-3c4f-41c6-b242-55149bf826b8 None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:47.508042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b9244-3c4f-41c6-b242-55149bf826b8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:47.509837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 69/142] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:47 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:47.516179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:47.655397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186787e-2897-4106-8fe8-2b988d7ddcf2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:47.680501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d469e-1dd2-408d-baea-75ef859b7344 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:47.681091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d469e-1dd2-408d-baea-75ef859b7344 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:47.681568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d469e-1dd2-408d-baea-75ef859b7344 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:47.683129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 70/143] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:47.695549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc32a97-92a8-4103-b2b1-7da820a2f254 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:47.696113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc32a97-92a8-4103-b2b1-7da820a2f254 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:47.696982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc32a97-92a8-4103-b2b1-7da820a2f254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:47.707604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ae970-de8c-46ad-9a88-e0b4475a0d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:47.708220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ae970-de8c-46ad-9a88-e0b4475a0d8b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:47.708615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ae970-de8c-46ad-9a88-e0b4475a0d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:47.708894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ae970-de8c-46ad-9a88-e0b4475a0d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:47.725193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bbc32a97-92a8-4103-b2b1-7da820a2f254 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:47.726515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bbc32a97-92a8-4103-b2b1-7da820a2f254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:47.849910 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:47.855722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 71/144] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:54:47.866267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ae970-de8c-46ad-9a88-e0b4475a0d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:47.867788 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ae970-de8c-46ad-9a88-e0b4475a0d8b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:48.009637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 74/145] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:48.015765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc32a97-92a8-4103-b2b1-7da820a2f254 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:48.131790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:48.162201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc32a97-92a8-4103-b2b1-7da820a2f254 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:48.210007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d68ca8-ee6f-4b72-9d9b-2a5b557d2784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:48.210636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68ca8-ee6f-4b72-9d9b-2a5b557d2784 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:48.210928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68ca8-ee6f-4b72-9d9b-2a5b557d2784 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:48.211219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68ca8-ee6f-4b72-9d9b-2a5b557d2784 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:48.212356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d68ca8-ee6f-4b72-9d9b-2a5b557d2784 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:48.213808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d68ca8-ee6f-4b72-9d9b-2a5b557d2784 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:48.215795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 72/146] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:48 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:48.222150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ae970-de8c-46ad-9a88-e0b4475a0d8b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:48.299076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d68ca8-ee6f-4b72-9d9b-2a5b557d2784 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:48.366888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:48.374123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ae970-de8c-46ad-9a88-e0b4475a0d8b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:48.439759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4f431-21c9-41e3-8e2a-eaa3ee40d53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:48.440370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f431-21c9-41e3-8e2a-eaa3ee40d53d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:48.440644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f431-21c9-41e3-8e2a-eaa3ee40d53d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:48.440909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4f431-21c9-41e3-8e2a-eaa3ee40d53d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:48.441699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4f431-21c9-41e3-8e2a-eaa3ee40d53d None admin] RBAC: Adding query filter params (role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=772dfdcbe95541519b7335820a928256, user.id=78837c5f7b204c9c96f712838f987797) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:48.442007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4f431-21c9-41e3-8e2a-eaa3ee40d53d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:48.443493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4f431-21c9-41e3-8e2a-eaa3ee40d53d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:48.448682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d68ca8-ee6f-4b72-9d9b-2a5b557d2784 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:48.454929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 75/147] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:48 2022] GET /identity/v3/role_assignments?user.id=78837c5f7b204c9c96f712838f987797&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=3b325dcac3064617a5415a84759ef71a => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:48.458066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2948-3c7e-41ed-b1e7-ca1bed01b841 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:48.458353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2948-3c7e-41ed-b1e7-ca1bed01b841 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:48.458623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2948-3c7e-41ed-b1e7-ca1bed01b841 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:48.459953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 76/148] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:48.467151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6705a51-c456-4b1f-9cc6-138da706dc78 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:48.467464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6705a51-c456-4b1f-9cc6-138da706dc78 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:48.467684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6705a51-c456-4b1f-9cc6-138da706dc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:48.508723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f6705a51-c456-4b1f-9cc6-138da706dc78 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:48.513561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f6705a51-c456-4b1f-9cc6-138da706dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:48.523434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce065df-3900-4bfe-9a9c-aad485d2db25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:48.524071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce065df-3900-4bfe-9a9c-aad485d2db25 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:48.524347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce065df-3900-4bfe-9a9c-aad485d2db25 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:48.524693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce065df-3900-4bfe-9a9c-aad485d2db25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:48.538151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce065df-3900-4bfe-9a9c-aad485d2db25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0ce065df-3900-4bfe-9a9c-aad485d2db25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:48.543430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:48.545486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 73/149] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:54:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:48.546254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:48.551577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce065df-3900-4bfe-9a9c-aad485d2db25 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:48.666988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce065df-3900-4bfe-9a9c-aad485d2db25 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:48.692643 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 77/150] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:48.705442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:48.706045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:48.706322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:48.706592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:48.707287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:48.707589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:48.709030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:48.721618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 74/151] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:54:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:48.723754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6705a51-c456-4b1f-9cc6-138da706dc78 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:48.839248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:48.852295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6705a51-c456-4b1f-9cc6-138da706dc78 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:48.890860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ee1c3-8527-4ef6-9cb7-510d40966218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:48.891519 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee1c3-8527-4ef6-9cb7-510d40966218 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:48.891794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee1c3-8527-4ef6-9cb7-510d40966218 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:48.892061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee1c3-8527-4ef6-9cb7-510d40966218 None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:48.893208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ee1c3-8527-4ef6-9cb7-510d40966218 None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:48.894717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ee1c3-8527-4ef6-9cb7-510d40966218 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:48.896386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 78/152] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:48 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:48.902282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:48.968700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:49.009697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f247b-1cb5-4516-87b0-50f01fbb8349 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:49.016990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a57e-e210-46df-854e-920852b42926 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.017275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a57e-e210-46df-854e-920852b42926 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.017550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a57e-e210-46df-854e-920852b42926 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.019061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 79/153] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:49.023843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bca4d-9a93-4dbd-906c-589bab99b58d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.024121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bca4d-9a93-4dbd-906c-589bab99b58d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.024392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bca4d-9a93-4dbd-906c-589bab99b58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.048370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-148bca4d-9a93-4dbd-906c-589bab99b58d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:49.049628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-148bca4d-9a93-4dbd-906c-589bab99b58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:49.099901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5e68b-c990-43bb-8bda-029d7bf3e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:49.100443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5e68b-c990-43bb-8bda-029d7bf3e8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.100723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5e68b-c990-43bb-8bda-029d7bf3e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.100997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5e68b-c990-43bb-8bda-029d7bf3e8e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.179373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 80/154] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:54:49.180471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bca4d-9a93-4dbd-906c-589bab99b58d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:49.230318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5e68b-c990-43bb-8bda-029d7bf3e8e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:49.231980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5e68b-c990-43bb-8bda-029d7bf3e8e3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:49.288926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:49.317899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bca4d-9a93-4dbd-906c-589bab99b58d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:49.349023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:49.391768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564a02f-abd5-414e-8465-7951b9dde47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:49.392399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564a02f-abd5-414e-8465-7951b9dde47f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.392593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564a02f-abd5-414e-8465-7951b9dde47f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.392865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564a02f-abd5-414e-8465-7951b9dde47f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.394021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564a02f-abd5-414e-8465-7951b9dde47f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:49.395768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564a02f-abd5-414e-8465-7951b9dde47f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:49.397839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 81/155] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:49.398806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d185db6-c7ac-4324-9df4-3aa3edc737c8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.399110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d185db6-c7ac-4324-9df4-3aa3edc737c8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.399382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d185db6-c7ac-4324-9df4-3aa3edc737c8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.400599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 82/156] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:49.405938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d185db6-c7ac-4324-9df4-3aa3edc737c8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:49.416349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 75/157] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:49.422978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caaad14-6f71-4b4d-9002-83d9e2af122d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.422978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caaad14-6f71-4b4d-9002-83d9e2af122d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.422978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caaad14-6f71-4b4d-9002-83d9e2af122d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.441712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5caaad14-6f71-4b4d-9002-83d9e2af122d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:49.443111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5caaad14-6f71-4b4d-9002-83d9e2af122d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:49.521348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d185db6-c7ac-4324-9df4-3aa3edc737c8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:49.521798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 76/158] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:49.522703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caaad14-6f71-4b4d-9002-83d9e2af122d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:49.563006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b117b1-00e7-4251-ad92-fc11e34ac38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:49.563714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b117b1-00e7-4251-ad92-fc11e34ac38c None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.563912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b117b1-00e7-4251-ad92-fc11e34ac38c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.564129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b117b1-00e7-4251-ad92-fc11e34ac38c None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.564755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b117b1-00e7-4251-ad92-fc11e34ac38c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:49.566182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b117b1-00e7-4251-ad92-fc11e34ac38c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:49.615075 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:49.624741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caaad14-6f71-4b4d-9002-83d9e2af122d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:49.658122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6919a-f746-46ee-b7da-7c2151fceffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:49.658694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6919a-f746-46ee-b7da-7c2151fceffe None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.658994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6919a-f746-46ee-b7da-7c2151fceffe None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.659270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6919a-f746-46ee-b7da-7c2151fceffe None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.660397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6919a-f746-46ee-b7da-7c2151fceffe None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:49.661844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6919a-f746-46ee-b7da-7c2151fceffe None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:49.663581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 77/159] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:49 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:49.663581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a1e5-8e7d-47e5-905e-242f9c8af5a6 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.663581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a1e5-8e7d-47e5-905e-242f9c8af5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.663581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a1e5-8e7d-47e5-905e-242f9c8af5a6 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.663581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 78/160] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:49.663581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a1e5-8e7d-47e5-905e-242f9c8af5a6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:49.670975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 83/161] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:54:49 2022] POST /identity/v3/users => generated 283 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:49.680190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b117b1-00e7-4251-ad92-fc11e34ac38c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:49.711544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:49.806527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a1e5-8e7d-47e5-905e-242f9c8af5a6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:49.842641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b117b1-00e7-4251-ad92-fc11e34ac38c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:49.847727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d283f-244a-45db-8175-21c6894e6867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:49.848373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d283f-244a-45db-8175-21c6894e6867 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.848653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d283f-244a-45db-8175-21c6894e6867 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.848925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d283f-244a-45db-8175-21c6894e6867 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.860440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d283f-244a-45db-8175-21c6894e6867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b90d283f-244a-45db-8175-21c6894e6867 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:49.865974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:49.866867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 79/162] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:49.867736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfb6f-443e-4c34-8103-83313fe1d183 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.868436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfb6f-443e-4c34-8103-83313fe1d183 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.868436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfb6f-443e-4c34-8103-83313fe1d183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.881290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:49.881898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:49.882172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:49.882435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:49.883202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] RBAC: Adding query filter params (role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=772dfdcbe95541519b7335820a928256, user.id=78837c5f7b204c9c96f712838f987797) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:49.883504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:49.884936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:49.890986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-992cfb6f-443e-4c34-8103-83313fe1d183 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:49.891822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-992cfb6f-443e-4c34-8103-83313fe1d183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:49.892583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 84/163] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:49 2022] GET /identity/v3/role_assignments?user.id=78837c5f7b204c9c96f712838f987797&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:49.899978 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:49.916466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:49.996560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 80/164] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:49.998520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cfb6f-443e-4c34-8103-83313fe1d183 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.024715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:50.072969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00daf17-a1ac-4f53-b8ad-830a5762c1f8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.113825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fa56e-ae09-4623-80f2-9a75037042b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.114827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fa56e-ae09-4623-80f2-9a75037042b6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.115345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fa56e-ae09-4623-80f2-9a75037042b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.115846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fa56e-ae09-4623-80f2-9a75037042b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.128490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fa56e-ae09-4623-80f2-9a75037042b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d90fa56e-ae09-4623-80f2-9a75037042b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:50.134358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:50.135272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:50.136066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:50.136743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cfb6f-443e-4c34-8103-83313fe1d183 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.139203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 85/165] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:54:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:50.145192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fa56e-ae09-4623-80f2-9a75037042b6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.177958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.178540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.178824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.179129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.180635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:50.180635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:50.181977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:50.195149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 81/166] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:50.201144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.214657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:50.264290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fa56e-ae09-4623-80f2-9a75037042b6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.301682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.302696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.303214 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.303691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.304690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:50.305193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:50.306783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:50.313520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b3543-9f73-4e86-9620-ac544df8cf58 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.316639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 86/167] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:50.319033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:50.324431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.360563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e84d28-0e80-4f1f-afba-0932bb8ae386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.361342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e84d28-0e80-4f1f-afba-0932bb8ae386 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.361719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e84d28-0e80-4f1f-afba-0932bb8ae386 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.362062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e84d28-0e80-4f1f-afba-0932bb8ae386 None admin] PATH_INFO: `/v3/users/6e1fabf461ea4de49e12baeb607e84d4` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.364526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e84d28-0e80-4f1f-afba-0932bb8ae386 None admin] RBAC: Authorizing `identity:get_user(user_id=6e1fabf461ea4de49e12baeb607e84d4)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:50.366100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e84d28-0e80-4f1f-afba-0932bb8ae386 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:50.368220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 82/168] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:50 2022] GET /identity/v3/users/6e1fabf461ea4de49e12baeb607e84d4 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:50.372879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e84d28-0e80-4f1f-afba-0932bb8ae386 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.452557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f414d6-af50-4185-975b-83a49a703845 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.476194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:50.493905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881ff70-e5d9-40b3-85cf-d54d42779415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.494784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881ff70-e5d9-40b3-85cf-d54d42779415 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.498386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881ff70-e5d9-40b3-85cf-d54d42779415 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.498702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881ff70-e5d9-40b3-85cf-d54d42779415 None admin] PATH_INFO: `/v3/users/70cd347e5af341fcb03bb3178f2c4247` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.499982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881ff70-e5d9-40b3-85cf-d54d42779415 None admin] RBAC: Authorizing `identity:get_user(user_id=70cd347e5af341fcb03bb3178f2c4247)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:50.501478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881ff70-e5d9-40b3-85cf-d54d42779415 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:50.503459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 87/169] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:50 2022] GET /identity/v3/users/70cd347e5af341fcb03bb3178f2c4247 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:50.504449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881ff70-e5d9-40b3-85cf-d54d42779415 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.506779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e84d28-0e80-4f1f-afba-0932bb8ae386 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.552480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cdf3c-a95e-46f8-9904-1b4f560a0707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.553101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cdf3c-a95e-46f8-9904-1b4f560a0707 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.553386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cdf3c-a95e-46f8-9904-1b4f560a0707 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.553657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cdf3c-a95e-46f8-9904-1b4f560a0707 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.566032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511cdf3c-a95e-46f8-9904-1b4f560a0707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-511cdf3c-a95e-46f8-9904-1b4f560a0707 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:50.572811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:50.579523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 83/170] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:50.580400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cdf3c-a95e-46f8-9904-1b4f560a0707 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.586966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:50.625536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881ff70-e5d9-40b3-85cf-d54d42779415 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.664945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f05ad6-dcc2-457f-a01f-9b493d6d44bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.665579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f05ad6-dcc2-457f-a01f-9b493d6d44bd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.665858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f05ad6-dcc2-457f-a01f-9b493d6d44bd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.666132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f05ad6-dcc2-457f-a01f-9b493d6d44bd None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.667344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f05ad6-dcc2-457f-a01f-9b493d6d44bd None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:50.669063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f05ad6-dcc2-457f-a01f-9b493d6d44bd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:50.672257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 88/171] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:50 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:50.673033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f05ad6-dcc2-457f-a01f-9b493d6d44bd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.789009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f05ad6-dcc2-457f-a01f-9b493d6d44bd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.821703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:50.827033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.828128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.828407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.828676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.829338 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:50.829639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:50.831099 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:50.840952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 89/172] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:50.842343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:50.867024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cdf3c-a95e-46f8-9904-1b4f560a0707 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:50.924241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e15871-312b-48a9-93e7-0b05703925c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:50.924853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e15871-312b-48a9-93e7-0b05703925c7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:50.925134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e15871-312b-48a9-93e7-0b05703925c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:50.925402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e15871-312b-48a9-93e7-0b05703925c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:50.936122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:51.012713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29182f78-5170-441b-ab9b-fa2b0ff7de13 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:51.035840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e15871-312b-48a9-93e7-0b05703925c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:51.037257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e15871-312b-48a9-93e7-0b05703925c7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:51.090184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36f1f8-40e1-4533-a8cb-27055b4e5bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:51.090871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36f1f8-40e1-4533-a8cb-27055b4e5bdc None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:51.091237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36f1f8-40e1-4533-a8cb-27055b4e5bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:51.091516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36f1f8-40e1-4533-a8cb-27055b4e5bdc None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256/users/6e1fabf461ea4de49e12baeb607e84d4/roles/976d0bc0f2b14f26877bcc529d3645c7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:51.093961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36f1f8-40e1-4533-a8cb-27055b4e5bdc None admin] RBAC: Authorizing `identity:create_grant(project_id=772dfdcbe95541519b7335820a928256, user_id=6e1fabf461ea4de49e12baeb607e84d4, role_id=976d0bc0f2b14f26877bcc529d3645c7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:51.095616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36f1f8-40e1-4533-a8cb-27055b4e5bdc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:51.153085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 84/173] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:51.153786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e15871-312b-48a9-93e7-0b05703925c7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:51.194601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:51.198252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 90/174] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:54:50 2022] PUT /identity/v3/projects/772dfdcbe95541519b7335820a928256/users/6e1fabf461ea4de49e12baeb607e84d4/roles/976d0bc0f2b14f26877bcc529d3645c7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:54:51.207282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36f1f8-40e1-4533-a8cb-27055b4e5bdc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:51.263736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e15871-312b-48a9-93e7-0b05703925c7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:51.301792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205128ae-fec7-4e48-afcb-6b77d74632f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:51.302363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205128ae-fec7-4e48-afcb-6b77d74632f4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:51.302761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205128ae-fec7-4e48-afcb-6b77d74632f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:51.303079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205128ae-fec7-4e48-afcb-6b77d74632f4 None admin] PATH_INFO: `/v3/users/78837c5f7b204c9c96f712838f987797` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:51.304358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205128ae-fec7-4e48-afcb-6b77d74632f4 None admin] RBAC: Authorizing `identity:get_user(user_id=78837c5f7b204c9c96f712838f987797)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:51.306122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:51.306449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205128ae-fec7-4e48-afcb-6b77d74632f4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:51.308391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 85/175] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:51 2022] GET /identity/v3/users/78837c5f7b204c9c96f712838f987797 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:51.313893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205128ae-fec7-4e48-afcb-6b77d74632f4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:51.385466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36f1f8-40e1-4533-a8cb-27055b4e5bdc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:51.446779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205128ae-fec7-4e48-afcb-6b77d74632f4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:51.452693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bad6c-325e-4499-9c0f-d6ab977f7370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:51.453347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bad6c-325e-4499-9c0f-d6ab977f7370 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:51.453672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bad6c-325e-4499-9c0f-d6ab977f7370 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:51.454095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bad6c-325e-4499-9c0f-d6ab977f7370 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:51.455491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bad6c-325e-4499-9c0f-d6ab977f7370 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:51.457011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bad6c-325e-4499-9c0f-d6ab977f7370 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:51.458708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 91/176] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:51 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:51.464552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bad6c-325e-4499-9c0f-d6ab977f7370 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:51.475360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:51.495733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899198a-d317-4520-a30a-d575cdeed5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:51.505270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899198a-d317-4520-a30a-d575cdeed5cc None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:51.505270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899198a-d317-4520-a30a-d575cdeed5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:51.505270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899198a-d317-4520-a30a-d575cdeed5cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:51.575086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899198a-d317-4520-a30a-d575cdeed5cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:51.576569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899198a-d317-4520-a30a-d575cdeed5cc None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:51.608095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bad6c-325e-4499-9c0f-d6ab977f7370 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:51.632610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:51.644798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:51.645409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:51.645690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:51.645962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:51.646691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=772dfdcbe95541519b7335820a928256, user.id=70cd347e5af341fcb03bb3178f2c4247) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:51.647038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:51.648886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:51.654876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 92/177] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:51 2022] GET /identity/v3/role_assignments?user.id=70cd347e5af341fcb03bb3178f2c4247&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:51.690328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 86/178] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:54:51.696666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:51.839114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd25ed-aca2-472c-883f-cb363ba6daa5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:51.877285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c22db-9ab3-4d74-80f4-a3ecccd6d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:51.877936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c22db-9ab3-4d74-80f4-a3ecccd6d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:51.878221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c22db-9ab3-4d74-80f4-a3ecccd6d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:51.878500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c22db-9ab3-4d74-80f4-a3ecccd6d1d0 None admin] PATH_INFO: `/v3/projects/5721358b3bfa4bc9898e7b80d7fb6299` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:51.879699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c22db-9ab3-4d74-80f4-a3ecccd6d1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=5721358b3bfa4bc9898e7b80d7fb6299)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:51.881325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c22db-9ab3-4d74-80f4-a3ecccd6d1d0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:51.883250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 93/179] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:51 2022] GET /identity/v3/projects/5721358b3bfa4bc9898e7b80d7fb6299 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:51.889781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899198a-d317-4520-a30a-d575cdeed5cc None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:51.903364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:52.004537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899198a-d317-4520-a30a-d575cdeed5cc None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:52.047464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec01b8-fa54-4483-a468-dec50383cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:52.048083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec01b8-fa54-4483-a468-dec50383cd99 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:52.048386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec01b8-fa54-4483-a468-dec50383cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:52.048659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec01b8-fa54-4483-a468-dec50383cd99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:52.049402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec01b8-fa54-4483-a468-dec50383cd99 None admin] RBAC: Adding query filter params (role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=5721358b3bfa4bc9898e7b80d7fb6299, user.id=78837c5f7b204c9c96f712838f987797) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:52.049714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec01b8-fa54-4483-a468-dec50383cd99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:52.051239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec01b8-fa54-4483-a468-dec50383cd99 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:52.057666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 87/180] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:51 2022] GET /identity/v3/role_assignments?user.id=78837c5f7b204c9c96f712838f987797&scope.project.id=5721358b3bfa4bc9898e7b80d7fb6299&role.id=3b325dcac3064617a5415a84759ef71a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:52.409946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aed77-3c64-43fe-b033-59a38c5de5a8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:52.410332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aed77-3c64-43fe-b033-59a38c5de5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:52.410516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aed77-3c64-43fe-b033-59a38c5de5a8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:52.412041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 94/181] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:52.417340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228d259-b808-4583-899b-48a10f9b6318 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:52.417630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228d259-b808-4583-899b-48a10f9b6318 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:52.417907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228d259-b808-4583-899b-48a10f9b6318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:52.437428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c228d259-b808-4583-899b-48a10f9b6318 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:52.438623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c228d259-b808-4583-899b-48a10f9b6318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:52.511594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:52.516841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 88/182] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:52.722934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca8bf-9ed0-4438-aa76-9ddbe34c361e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:52.723518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca8bf-9ed0-4438-aa76-9ddbe34c361e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:52.723603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca8bf-9ed0-4438-aa76-9ddbe34c361e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:52.725198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 95/183] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:52.731529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906e228-0731-4bbd-8e9d-cfca5a15e65b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:52.732066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906e228-0731-4bbd-8e9d-cfca5a15e65b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:52.732158 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906e228-0731-4bbd-8e9d-cfca5a15e65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:52.750255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2906e228-0731-4bbd-8e9d-cfca5a15e65b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:52.751473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2906e228-0731-4bbd-8e9d-cfca5a15e65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:52.823563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca8bf-9ed0-4438-aa76-9ddbe34c361e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:52.824066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 89/184] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:52.888970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:52.932286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca8bf-9ed0-4438-aa76-9ddbe34c361e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:52.967670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1775b-780f-4259-8e67-f74c2a9daf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:52.968299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1775b-780f-4259-8e67-f74c2a9daf8d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:52.968580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1775b-780f-4259-8e67-f74c2a9daf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:52.968853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1775b-780f-4259-8e67-f74c2a9daf8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:52.979503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1775b-780f-4259-8e67-f74c2a9daf8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e0c1775b-780f-4259-8e67-f74c2a9daf8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:52.985305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:52.986199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:52.987563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 96/185] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:54:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:52.992235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906e228-0731-4bbd-8e9d-cfca5a15e65b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.006541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df1dfa-90c0-4f9b-a33e-e30f39ca3415 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.006820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df1dfa-90c0-4f9b-a33e-e30f39ca3415 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.007171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df1dfa-90c0-4f9b-a33e-e30f39ca3415 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.008524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 97/186] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:53.013776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd627e-7ffd-4665-ae1e-9dbb1f1da41a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.014066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd627e-7ffd-4665-ae1e-9dbb1f1da41a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.014339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd627e-7ffd-4665-ae1e-9dbb1f1da41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.032369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cfdd627e-7ffd-4665-ae1e-9dbb1f1da41a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:53.033570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cfdd627e-7ffd-4665-ae1e-9dbb1f1da41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:53.109373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 98/187] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:54:53.123880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906e228-0731-4bbd-8e9d-cfca5a15e65b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.126897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd627e-7ffd-4665-ae1e-9dbb1f1da41a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.158521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:53.164797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.165455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.165745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.166017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.166715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:53.167056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:53.168434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:53.177505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:53.178575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 90/188] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:53.188309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.242363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd627e-7ffd-4665-ae1e-9dbb1f1da41a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.281114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a782-0911-4539-85f7-7d9eea6ad704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.282118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a782-0911-4539-85f7-7d9eea6ad704 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.282542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a782-0911-4539-85f7-7d9eea6ad704 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.282991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a782-0911-4539-85f7-7d9eea6ad704 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.293993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a782-0911-4539-85f7-7d9eea6ad704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0194a782-0911-4539-85f7-7d9eea6ad704 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:53.299473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:53.300601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:53.301400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:53.303987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 99/189] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:54:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:53.315011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508599ff-8918-4731-919b-612e2046bce0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.315202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a782-0911-4539-85f7-7d9eea6ad704 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.360488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edf189-79da-4aeb-8c94-53b489da5efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.361377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edf189-79da-4aeb-8c94-53b489da5efe None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.361799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edf189-79da-4aeb-8c94-53b489da5efe None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.362212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edf189-79da-4aeb-8c94-53b489da5efe None admin] PATH_INFO: `/v3/users/20ead75f8301462fa4c8e66b4f142446` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.376381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edf189-79da-4aeb-8c94-53b489da5efe None admin] RBAC: Authorizing `identity:get_user(user_id=20ead75f8301462fa4c8e66b4f142446)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:53.378073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edf189-79da-4aeb-8c94-53b489da5efe None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:53.379914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 91/190] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:53 2022] GET /identity/v3/users/20ead75f8301462fa4c8e66b4f142446 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:53.385514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edf189-79da-4aeb-8c94-53b489da5efe None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.437988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:53.438401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a782-0911-4539-85f7-7d9eea6ad704 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.441116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:53.474076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.474679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.474999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.475294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.475962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:53.476265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:53.477661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:53.490871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 100/191] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:54:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:53.490871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.494516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edf189-79da-4aeb-8c94-53b489da5efe None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.531127 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9d59d-d66d-4621-85c6-28280d43e342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.531750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9d59d-d66d-4621-85c6-28280d43e342 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.532218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9d59d-d66d-4621-85c6-28280d43e342 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.532497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9d59d-d66d-4621-85c6-28280d43e342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.592395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53f94b-f558-4012-acf2-fa2d60c3d344 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.602720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9d59d-d66d-4621-85c6-28280d43e342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:53.604235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9d59d-d66d-4621-85c6-28280d43e342 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:53.626582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576408ee-8afc-416a-ac3e-562ad7ecaa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.627229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576408ee-8afc-416a-ac3e-562ad7ecaa1b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.627508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576408ee-8afc-416a-ac3e-562ad7ecaa1b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.627784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576408ee-8afc-416a-ac3e-562ad7ecaa1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.628880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576408ee-8afc-416a-ac3e-562ad7ecaa1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:53.630301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576408ee-8afc-416a-ac3e-562ad7ecaa1b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:53.636334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 101/192] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:53.636334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576408ee-8afc-416a-ac3e-562ad7ecaa1b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.652367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:53.668341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:53.703357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 92/193] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:53.708799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493935-ac29-48d6-a85b-a52a5d270205 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.708799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493935-ac29-48d6-a85b-a52a5d270205 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.708799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493935-ac29-48d6-a85b-a52a5d270205 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.708799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 93/194] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:53.708799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9493935-ac29-48d6-a85b-a52a5d270205 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.736751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576408ee-8afc-416a-ac3e-562ad7ecaa1b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.768735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c019c24-fc6a-439a-8a43-30f762f30466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.769292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c019c24-fc6a-439a-8a43-30f762f30466 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.769571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c019c24-fc6a-439a-8a43-30f762f30466 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.769841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c019c24-fc6a-439a-8a43-30f762f30466 None admin] PATH_INFO: `/v3/users/20ead75f8301462fa4c8e66b4f142446` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.770985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c019c24-fc6a-439a-8a43-30f762f30466 None admin] RBAC: Authorizing `identity:get_user(user_id=20ead75f8301462fa4c8e66b4f142446)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:53.772432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c019c24-fc6a-439a-8a43-30f762f30466 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:53.774160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 102/195] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:53 2022] GET /identity/v3/users/20ead75f8301462fa4c8e66b4f142446 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:53.774755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c019c24-fc6a-439a-8a43-30f762f30466 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.810128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9493935-ac29-48d6-a85b-a52a5d270205 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.844524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedba77-7bfb-4f0e-8f1d-1889caa1fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.845217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedba77-7bfb-4f0e-8f1d-1889caa1fc57 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.845551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedba77-7bfb-4f0e-8f1d-1889caa1fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.845883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedba77-7bfb-4f0e-8f1d-1889caa1fc57 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.846576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedba77-7bfb-4f0e-8f1d-1889caa1fc57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:53.848106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fedba77-7bfb-4f0e-8f1d-1889caa1fc57 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:53.873174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c019c24-fc6a-439a-8a43-30f762f30466 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:53.874690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:53.906264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30f9dd-1b05-4099-bf13-a01a71c53f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:53.906879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30f9dd-1b05-4099-bf13-a01a71c53f97 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.908334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30f9dd-1b05-4099-bf13-a01a71c53f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.908334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30f9dd-1b05-4099-bf13-a01a71c53f97 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.908591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30f9dd-1b05-4099-bf13-a01a71c53f97 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:53.910012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30f9dd-1b05-4099-bf13-a01a71c53f97 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:53.911649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 103/196] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:53 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:53.912718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6daecb-e6e8-47ab-add6-b10bab62a7a4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:53.912995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6daecb-e6e8-47ab-add6-b10bab62a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:53.913261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6daecb-e6e8-47ab-add6-b10bab62a7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:53.929870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0f6daecb-e6e8-47ab-add6-b10bab62a7a4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:53.931075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0f6daecb-e6e8-47ab-add6-b10bab62a7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:53.943083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 94/197] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:54:53 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:53.943083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedba77-7bfb-4f0e-8f1d-1889caa1fc57 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:53.943732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:54.011025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 104/198] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:54.011025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6daecb-e6e8-47ab-add6-b10bab62a7a4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:54.070980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedba77-7bfb-4f0e-8f1d-1889caa1fc57 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:54.118674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbb8c5-e69f-4f15-bbea-8c61775e4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:54.119755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbb8c5-e69f-4f15-bbea-8c61775e4009 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:54.120093 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbb8c5-e69f-4f15-bbea-8c61775e4009 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:54.120373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbb8c5-e69f-4f15-bbea-8c61775e4009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:54.148862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6daecb-e6e8-47ab-add6-b10bab62a7a4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:54.150690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:54.195541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:54.196141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:54.196415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:54.196940 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:54.197664 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=5749ebcaf01649d2a355e1191671ad21, user.id=20ead75f8301462fa4c8e66b4f142446) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:54.197965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:54.199617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:54.210969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 105/199] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:54 2022] GET /identity/v3/role_assignments?user.id=20ead75f8301462fa4c8e66b4f142446&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:54.223811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbb8c5-e69f-4f15-bbea-8c61775e4009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:54.225338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbb8c5-e69f-4f15-bbea-8c61775e4009 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:54.235307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:54.275271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:54.355579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 95/200] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:54.367028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbb8c5-e69f-4f15-bbea-8c61775e4009 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:54.414749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4dce7-a5c1-4eb7-bacb-f47d59d1c0b1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:54.453125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d964ce-4732-4945-b946-e3884bb7adbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:54.453772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d964ce-4732-4945-b946-e3884bb7adbb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:54.454110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d964ce-4732-4945-b946-e3884bb7adbb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:54.454399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d964ce-4732-4945-b946-e3884bb7adbb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:54.474341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d964ce-4732-4945-b946-e3884bb7adbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-83d964ce-4732-4945-b946-e3884bb7adbb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:54.479672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:54.480567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:54.481376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 106/201] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:54.487589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbb8c5-e69f-4f15-bbea-8c61775e4009 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:54.488530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d964ce-4732-4945-b946-e3884bb7adbb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:54.495760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:54.530098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14cb79-18bc-4fd7-945d-7aff5b9de8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:54.530743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14cb79-18bc-4fd7-945d-7aff5b9de8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:54.531035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14cb79-18bc-4fd7-945d-7aff5b9de8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:54.531302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14cb79-18bc-4fd7-945d-7aff5b9de8a8 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:54.536475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14cb79-18bc-4fd7-945d-7aff5b9de8a8 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:54.539779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14cb79-18bc-4fd7-945d-7aff5b9de8a8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:54.541639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 96/202] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:54 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:54.548197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:54.564942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14cb79-18bc-4fd7-945d-7aff5b9de8a8 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:54.677408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14cb79-18bc-4fd7-945d-7aff5b9de8a8 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:54.719763 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:54.720449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:54.720724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:54.721025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:54.721762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] RBAC: Adding query filter params (role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=5749ebcaf01649d2a355e1191671ad21, user.id=20ead75f8301462fa4c8e66b4f142446) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:54.722097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:54.726964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:54.731298 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 97/203] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:54 2022] GET /identity/v3/role_assignments?user.id=20ead75f8301462fa4c8e66b4f142446&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:54.769407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d964ce-4732-4945-b946-e3884bb7adbb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:54.840717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:54.841394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:54.841670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:54.841938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:54.842616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:54.842919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:54.844366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:54.855139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 107/204] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:54.861333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:54.920686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:54.998921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e387d-5e58-4ca4-8564-1423e558625c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:55.035623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99713c6-8ca9-4157-b197-5dd6fc8a8c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:55.036535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99713c6-8ca9-4157-b197-5dd6fc8a8c18 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:55.037007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99713c6-8ca9-4157-b197-5dd6fc8a8c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:55.037490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99713c6-8ca9-4157-b197-5dd6fc8a8c18 None admin] PATH_INFO: `/v3/users/6e1fabf461ea4de49e12baeb607e84d4` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:55.038796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99713c6-8ca9-4157-b197-5dd6fc8a8c18 None admin] RBAC: Authorizing `identity:get_user(user_id=6e1fabf461ea4de49e12baeb607e84d4)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:55.040512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99713c6-8ca9-4157-b197-5dd6fc8a8c18 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:55.042369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 98/205] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:54 2022] GET /identity/v3/users/6e1fabf461ea4de49e12baeb607e84d4 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:55.048716 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:55.174206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:55.201987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f17a-64be-40aa-ba84-b85e66d672cc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:55.251710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf22d7-f9b0-4d03-86ec-2238e6a23362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:55.253567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf22d7-f9b0-4d03-86ec-2238e6a23362 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:55.253845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf22d7-f9b0-4d03-86ec-2238e6a23362 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:55.254115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf22d7-f9b0-4d03-86ec-2238e6a23362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:55.353037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf22d7-f9b0-4d03-86ec-2238e6a23362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:55.354507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf22d7-f9b0-4d03-86ec-2238e6a23362 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:55.471720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:55.493147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 108/206] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:54:55.499108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99713c6-8ca9-4157-b197-5dd6fc8a8c18 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:55.665477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99713c6-8ca9-4157-b197-5dd6fc8a8c18 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:55.673025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:55.700914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b291c-52f4-485d-8810-7a5952316166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:55.701897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b291c-52f4-485d-8810-7a5952316166 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:55.702368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b291c-52f4-485d-8810-7a5952316166 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:55.702831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b291c-52f4-485d-8810-7a5952316166 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:55.704184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b291c-52f4-485d-8810-7a5952316166 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:55.705851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b291c-52f4-485d-8810-7a5952316166 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:55.710968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 99/207] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:55 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:55.714056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf22d7-f9b0-4d03-86ec-2238e6a23362 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:55.857572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf22d7-f9b0-4d03-86ec-2238e6a23362 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:55.909031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31f726-7560-422c-88fb-4aa2881c3c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:55.909637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31f726-7560-422c-88fb-4aa2881c3c07 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:55.909911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31f726-7560-422c-88fb-4aa2881c3c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:55.910180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31f726-7560-422c-88fb-4aa2881c3c07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:55.910903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31f726-7560-422c-88fb-4aa2881c3c07 None admin] RBAC: Adding query filter params (role.id=976d0bc0f2b14f26877bcc529d3645c7, scope.project.id=772dfdcbe95541519b7335820a928256, user.id=6e1fabf461ea4de49e12baeb607e84d4) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:55.911231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31f726-7560-422c-88fb-4aa2881c3c07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:55.912662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c31f726-7560-422c-88fb-4aa2881c3c07 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:55.923660 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 109/208] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:55 2022] GET /identity/v3/role_assignments?user.id=6e1fabf461ea4de49e12baeb607e84d4&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=976d0bc0f2b14f26877bcc529d3645c7 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:56.364309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789fc7-9a5a-45b1-a9b4-bd4b9c15fa7f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:56.364785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789fc7-9a5a-45b1-a9b4-bd4b9c15fa7f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:56.364785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789fc7-9a5a-45b1-a9b4-bd4b9c15fa7f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:56.366246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 100/209] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:56.371606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdadf7c-cfc0-4ab0-b87a-29646cad9198 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:56.371889 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdadf7c-cfc0-4ab0-b87a-29646cad9198 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:56.372240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdadf7c-cfc0-4ab0-b87a-29646cad9198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:56.394207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-abdadf7c-cfc0-4ab0-b87a-29646cad9198 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:56.395409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-abdadf7c-cfc0-4ab0-b87a-29646cad9198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:56.430222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb327b3-7024-4e6d-b614-b95b47573638 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:56.430491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb327b3-7024-4e6d-b614-b95b47573638 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:56.430744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb327b3-7024-4e6d-b614-b95b47573638 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:56.432239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 101/210] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:54:56.444316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4fba3-672a-467b-9fba-1605ced89402 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:56.444588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4fba3-672a-467b-9fba-1605ced89402 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:56.444862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4fba3-672a-467b-9fba-1605ced89402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:56.446757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:56.466461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-14b4fba3-672a-467b-9fba-1605ced89402 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:56.470423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-14b4fba3-672a-467b-9fba-1605ced89402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:56.476620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 110/211] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:56.545033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 102/212] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:54:56.560645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf619e-7c8e-4154-94a0-c904c52c6444 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:56.560957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf619e-7c8e-4154-94a0-c904c52c6444 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:56.561201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf619e-7c8e-4154-94a0-c904c52c6444 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:56.562884 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 111/213] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:56.568220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598638-6c69-4988-b926-7a0fb9b0fab7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:56.568504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598638-6c69-4988-b926-7a0fb9b0fab7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:56.568778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598638-6c69-4988-b926-7a0fb9b0fab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:56.628998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2f598638-6c69-4988-b926-7a0fb9b0fab7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:56.630110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2f598638-6c69-4988-b926-7a0fb9b0fab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:56.699215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 103/214] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:56.777040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf619e-7c8e-4154-94a0-c904c52c6444 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:56.821904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598638-6c69-4988-b926-7a0fb9b0fab7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:56.857998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:56.897749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf619e-7c8e-4154-94a0-c904c52c6444 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:56.937932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bf81e-3e2b-440d-aba9-fa7739baac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:56.938850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bf81e-3e2b-440d-aba9-fa7739baac4b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:56.939327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bf81e-3e2b-440d-aba9-fa7739baac4b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:56.939805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bf81e-3e2b-440d-aba9-fa7739baac4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:56.940071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598638-6c69-4988-b926-7a0fb9b0fab7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:56.952709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bf81e-3e2b-440d-aba9-fa7739baac4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-002bf81e-3e2b-440d-aba9-fa7739baac4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:56.958078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:56.959215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:54:56.960082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:56.967037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 112/215] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:54:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:56.968068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bf81e-3e2b-440d-aba9-fa7739baac4b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:56.978337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf765f3f-bd61-4b8e-8a42-e7b1aa610923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:56.979056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765f3f-bd61-4b8e-8a42-e7b1aa610923 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:56.979388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765f3f-bd61-4b8e-8a42-e7b1aa610923 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:56.979663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765f3f-bd61-4b8e-8a42-e7b1aa610923 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:56.981193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf765f3f-bd61-4b8e-8a42-e7b1aa610923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:56.983325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf765f3f-bd61-4b8e-8a42-e7b1aa610923 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:56.984903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 104/216] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:54:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:56.985664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf765f3f-bd61-4b8e-8a42-e7b1aa610923 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.015745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:57.088772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bf81e-3e2b-440d-aba9-fa7739baac4b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.090226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf765f3f-bd61-4b8e-8a42-e7b1aa610923 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.092254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:57.124918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.125596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.125882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.126153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.126658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd426ab-e3ae-4de8-b1ef-3485a77f7fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.126818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:57.127153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:57.127289 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd426ab-e3ae-4de8-b1ef-3485a77f7fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.127570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd426ab-e3ae-4de8-b1ef-3485a77f7fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.127840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd426ab-e3ae-4de8-b1ef-3485a77f7fa2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.128537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:57.137656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 113/217] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:57.137656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.143332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd426ab-e3ae-4de8-b1ef-3485a77f7fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-fdd426ab-e3ae-4de8-b1ef-3485a77f7fa2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:57.149597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:57.150665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:54:57.151607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:54:57.154493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 105/218] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:54:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:54:57.155355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd426ab-e3ae-4de8-b1ef-3485a77f7fa2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.245529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4e848-be14-46db-aaa5-256c2da25509 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.247035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:57.254006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd426ab-e3ae-4de8-b1ef-3485a77f7fa2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.280598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ce152-4742-493b-a7ba-41995754f145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.281210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce152-4742-493b-a7ba-41995754f145 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.281487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce152-4742-493b-a7ba-41995754f145 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.281761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce152-4742-493b-a7ba-41995754f145 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.282400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ce152-4742-493b-a7ba-41995754f145 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:57.283881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ce152-4742-493b-a7ba-41995754f145 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:57.289186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced21a-efa7-4840-9066-2b13a4356c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.289816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced21a-efa7-4840-9066-2b13a4356c36 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.290094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced21a-efa7-4840-9066-2b13a4356c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.290366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced21a-efa7-4840-9066-2b13a4356c36 None admin] PATH_INFO: `/v3/users/20ead75f8301462fa4c8e66b4f142446` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.291535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced21a-efa7-4840-9066-2b13a4356c36 None admin] RBAC: Authorizing `identity:get_user(user_id=20ead75f8301462fa4c8e66b4f142446)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:57.292986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eced21a-efa7-4840-9066-2b13a4356c36 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:57.295117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 106/219] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:57 2022] GET /identity/v3/users/20ead75f8301462fa4c8e66b4f142446 => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:57.295873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced21a-efa7-4840-9066-2b13a4356c36 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.325795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:57.382471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 114/220] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:54:57 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:54:57.383985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f954a-3dd9-4833-9f89-94587d266a43 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.384274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f954a-3dd9-4833-9f89-94587d266a43 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.384553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f954a-3dd9-4833-9f89-94587d266a43 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.385847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 115/221] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:57.386572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f954a-3dd9-4833-9f89-94587d266a43 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.395916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced21a-efa7-4840-9066-2b13a4356c36 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.438130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8897d1-9e8f-4d6c-a815-d427292bf507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.438130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8897d1-9e8f-4d6c-a815-d427292bf507 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.438458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8897d1-9e8f-4d6c-a815-d427292bf507 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.438795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8897d1-9e8f-4d6c-a815-d427292bf507 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.439511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8897d1-9e8f-4d6c-a815-d427292bf507 None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:57.448420 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8897d1-9e8f-4d6c-a815-d427292bf507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:57.448420 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8897d1-9e8f-4d6c-a815-d427292bf507 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:57.455803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 107/222] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:54:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:57.455803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b2b3a-5a70-4ab4-b425-bf3b990472bb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.455803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b2b3a-5a70-4ab4-b425-bf3b990472bb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.455803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b2b3a-5a70-4ab4-b425-bf3b990472bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.472974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b96b2b3a-5a70-4ab4-b425-bf3b990472bb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:57.474203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b96b2b3a-5a70-4ab4-b425-bf3b990472bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:57.530734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f954a-3dd9-4833-9f89-94587d266a43 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.556740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 108/223] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:57.557612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b2b3a-5a70-4ab4-b425-bf3b990472bb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.573570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82137e-971a-45a9-b037-c137490df670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.574198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82137e-971a-45a9-b037-c137490df670 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.574480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82137e-971a-45a9-b037-c137490df670 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.574754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82137e-971a-45a9-b037-c137490df670 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.576142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82137e-971a-45a9-b037-c137490df670 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:57.578065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82137e-971a-45a9-b037-c137490df670 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:57.581965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 116/224] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:57 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:57.585450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82137e-971a-45a9-b037-c137490df670 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.592282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:57.621771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:57.685437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b2b3a-5a70-4ab4-b425-bf3b990472bb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.724549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82137e-971a-45a9-b037-c137490df670 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.738724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c728-354e-43cf-8cfc-034b7f954554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.739837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c728-354e-43cf-8cfc-034b7f954554 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.740475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c728-354e-43cf-8cfc-034b7f954554 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.741019 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c728-354e-43cf-8cfc-034b7f954554 None admin] PATH_INFO: `/v3/users/493803e3313a47009f6071ad408a9a6e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.759208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c728-354e-43cf-8cfc-034b7f954554 None admin] RBAC: Authorizing `identity:get_user(user_id=493803e3313a47009f6071ad408a9a6e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:57.761554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c728-354e-43cf-8cfc-034b7f954554 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:57.765100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 109/225] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:57 2022] GET /identity/v3/users/493803e3313a47009f6071ad408a9a6e => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:54:57.771560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c728-354e-43cf-8cfc-034b7f954554 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.777327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ad0f7-b15b-4df1-b983-e646529de902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.778071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ad0f7-b15b-4df1-b983-e646529de902 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.778489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ad0f7-b15b-4df1-b983-e646529de902 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.778834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ad0f7-b15b-4df1-b983-e646529de902 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21/users/20ead75f8301462fa4c8e66b4f142446/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:57.784274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ad0f7-b15b-4df1-b983-e646529de902 None admin] RBAC: Authorizing `identity:create_grant(project_id=5749ebcaf01649d2a355e1191671ad21, user_id=20ead75f8301462fa4c8e66b4f142446, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:57.790963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ad0f7-b15b-4df1-b983-e646529de902 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:57.858885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 117/226] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:54:57 2022] PUT /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21/users/20ead75f8301462fa4c8e66b4f142446/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:54:57.860933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ad0f7-b15b-4df1-b983-e646529de902 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:57.880767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:57.907343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c728-354e-43cf-8cfc-034b7f954554 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:57.993145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5c476-1ef3-40a8-866e-622f62b33022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:57.996928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c476-1ef3-40a8-866e-622f62b33022 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:57.996928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c476-1ef3-40a8-866e-622f62b33022 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:57.996928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c476-1ef3-40a8-866e-622f62b33022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:58.022172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:58.053071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ad0f7-b15b-4df1-b983-e646529de902 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:58.103392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5c476-1ef3-40a8-866e-622f62b33022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:58.105501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5c476-1ef3-40a8-866e-622f62b33022 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:58.107695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cef32-f373-4378-97a2-b62dff85dfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:58.108437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cef32-f373-4378-97a2-b62dff85dfff None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:58.108742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cef32-f373-4378-97a2-b62dff85dfff None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:58.109084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cef32-f373-4378-97a2-b62dff85dfff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:58.120730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cef32-f373-4378-97a2-b62dff85dfff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1b8cef32-f373-4378-97a2-b62dff85dfff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:54:58.134931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:54:58.137108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:54:58.139162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 118/227] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:54:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:54:58.142973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cef32-f373-4378-97a2-b62dff85dfff None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:58.208479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:58.219921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 110/228] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:54:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:54:58.226501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5c476-1ef3-40a8-866e-622f62b33022 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:58.353888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cef32-f373-4378-97a2-b62dff85dfff None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:58.419693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:58.420302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:58.420578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:58.420846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:58.421512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:58.421813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:58.423231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:58.426295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5c476-1ef3-40a8-866e-622f62b33022 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:58.435720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 119/229] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:54:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:58.441229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:58.452614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:58.501956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81661a8-08ff-4b5d-bb0e-b6a8967c0017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:58.502930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81661a8-08ff-4b5d-bb0e-b6a8967c0017 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:58.503406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81661a8-08ff-4b5d-bb0e-b6a8967c0017 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:58.503886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81661a8-08ff-4b5d-bb0e-b6a8967c0017 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:58.506175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81661a8-08ff-4b5d-bb0e-b6a8967c0017 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:58.507858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81661a8-08ff-4b5d-bb0e-b6a8967c0017 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:58.510190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 111/230] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:58 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:58.516183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81661a8-08ff-4b5d-bb0e-b6a8967c0017 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:58.545504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c1d8d-e567-4059-b4dc-71919928ad1c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:58.600148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b02a-6876-401a-95fc-a15a7075e326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:58.602030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b02a-6876-401a-95fc-a15a7075e326 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:58.602314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b02a-6876-401a-95fc-a15a7075e326 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:58.602589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507b02a-6876-401a-95fc-a15a7075e326 None admin] PATH_INFO: `/v3/users/20ead75f8301462fa4c8e66b4f142446` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:58.604691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507b02a-6876-401a-95fc-a15a7075e326 None admin] RBAC: Authorizing `identity:get_user(user_id=20ead75f8301462fa4c8e66b4f142446)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:58.606081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507b02a-6876-401a-95fc-a15a7075e326 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:58.607896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 120/231] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:54:58 2022] GET /identity/v3/users/20ead75f8301462fa4c8e66b4f142446 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:58.613493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b02a-6876-401a-95fc-a15a7075e326 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:58.615306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:58.714755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81661a8-08ff-4b5d-bb0e-b6a8967c0017 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:58.765675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:58.787065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:58.787717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:58.788050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:58.788324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:58.789132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] RBAC: Adding query filter params (role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=5749ebcaf01649d2a355e1191671ad21, user.id=493803e3313a47009f6071ad408a9a6e) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:54:58.789449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:58.790912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:58.802572 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 112/232] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:54:58 2022] GET /identity/v3/role_assignments?user.id=493803e3313a47009f6071ad408a9a6e&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=3b325dcac3064617a5415a84759ef71a => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:58.815820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507b02a-6876-401a-95fc-a15a7075e326 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:58.874861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f3ac4-66e6-47b1-a6a5-8af29252d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:58.875512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f3ac4-66e6-47b1-a6a5-8af29252d40e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:58.875796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f3ac4-66e6-47b1-a6a5-8af29252d40e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:58.876072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f3ac4-66e6-47b1-a6a5-8af29252d40e None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:58.877254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f3ac4-66e6-47b1-a6a5-8af29252d40e None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:58.878766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f3ac4-66e6-47b1-a6a5-8af29252d40e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:58.880803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 121/233] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:54:58 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:54:58.886471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:54:59.031075 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4af9-98cd-4bce-8b1b-91863a8f0434 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:54:59.044103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:54:59.072297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0761c64-b455-444d-ba78-05b950b41c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:54:59.072959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0761c64-b455-444d-ba78-05b950b41c1e None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:59.073233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0761c64-b455-444d-ba78-05b950b41c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:59.073510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0761c64-b455-444d-ba78-05b950b41c1e None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21/users/20ead75f8301462fa4c8e66b4f142446/roles/af615e7795ae42019a45dfe2bb205c49` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:59.080793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0761c64-b455-444d-ba78-05b950b41c1e None admin] RBAC: Authorizing `identity:create_grant(project_id=5749ebcaf01649d2a355e1191671ad21, user_id=20ead75f8301462fa4c8e66b4f142446, role_id=af615e7795ae42019a45dfe2bb205c49)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:54:59.082242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0761c64-b455-444d-ba78-05b950b41c1e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:54:59.168718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 113/234] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:54:58 2022] PUT /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21/users/20ead75f8301462fa4c8e66b4f142446/roles/af615e7795ae42019a45dfe2bb205c49 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:54:59.806703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18201bd-989d-443e-984d-9d1d178cd185 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:59.807372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18201bd-989d-443e-984d-9d1d178cd185 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:59.807372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18201bd-989d-443e-984d-9d1d178cd185 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:59.808635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 122/235] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:59.827795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405742c-9d73-48bf-a48c-ee985e06e715 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:59.828710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405742c-9d73-48bf-a48c-ee985e06e715 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:59.829221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405742c-9d73-48bf-a48c-ee985e06e715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:59.856004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a405742c-9d73-48bf-a48c-ee985e06e715 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:54:59.857624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a405742c-9d73-48bf-a48c-ee985e06e715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:54:59.941163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 114/236] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:54:59.983183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770268fa-e572-49c9-82b4-e086387f3fda None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:59.983520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770268fa-e572-49c9-82b4-e086387f3fda None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:59.983743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770268fa-e572-49c9-82b4-e086387f3fda None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:54:59.985348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 123/237] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:54:59.990880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63810bda-c405-43ae-a720-d687325173b5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:54:59.991210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63810bda-c405-43ae-a720-d687325173b5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:54:59.991480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63810bda-c405-43ae-a720-d687325173b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:00.015277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-63810bda-c405-43ae-a720-d687325173b5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:00.016472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-63810bda-c405-43ae-a720-d687325173b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:00.103408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 115/238] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:00.213587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201cedb-d845-49ba-bfc1-5784c49e34bd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:00.213906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201cedb-d845-49ba-bfc1-5784c49e34bd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:00.214164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201cedb-d845-49ba-bfc1-5784c49e34bd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:00.215541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 124/239] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:00.221051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88c63-a70e-48e0-8f65-7f43b1cbfc29 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:00.221340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88c63-a70e-48e0-8f65-7f43b1cbfc29 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:00.221609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a88c63-a70e-48e0-8f65-7f43b1cbfc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:00.243239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-95a88c63-a70e-48e0-8f65-7f43b1cbfc29 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:00.244414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-95a88c63-a70e-48e0-8f65-7f43b1cbfc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:00.282145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:00.336250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 116/240] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:00.432908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201cedb-d845-49ba-bfc1-5784c49e34bd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:00.474348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88c63-a70e-48e0-8f65-7f43b1cbfc29 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:00.607350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a88c63-a70e-48e0-8f65-7f43b1cbfc29 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:00.620463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201cedb-d845-49ba-bfc1-5784c49e34bd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:00.652471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14655aff-db30-4aec-ad7a-4b05d74cb60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:00.654540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655aff-db30-4aec-ad7a-4b05d74cb60e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:00.654889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655aff-db30-4aec-ad7a-4b05d74cb60e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:00.655202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655aff-db30-4aec-ad7a-4b05d74cb60e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:00.656482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14655aff-db30-4aec-ad7a-4b05d74cb60e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:00.657949 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14655aff-db30-4aec-ad7a-4b05d74cb60e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:00.659594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 117/241] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:00.660360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14655aff-db30-4aec-ad7a-4b05d74cb60e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:00.679046 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216b43-f9c2-471e-9628-d776227c76f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:00.680239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216b43-f9c2-471e-9628-d776227c76f4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:00.681189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216b43-f9c2-471e-9628-d776227c76f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:00.681659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216b43-f9c2-471e-9628-d776227c76f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:00.696326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:00.697177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff216b43-f9c2-471e-9628-d776227c76f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ff216b43-f9c2-471e-9628-d776227c76f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:00.704646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:00.710619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 125/242] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:00.711908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216b43-f9c2-471e-9628-d776227c76f4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:00.715160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:00.775570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14655aff-db30-4aec-ad7a-4b05d74cb60e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:00.813325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38a76c-8bde-49b4-a75e-d592294df81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:00.814331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a76c-8bde-49b4-a75e-d592294df81d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:00.814895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a76c-8bde-49b4-a75e-d592294df81d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:00.815438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a76c-8bde-49b4-a75e-d592294df81d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:00.828581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38a76c-8bde-49b4-a75e-d592294df81d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-1a38a76c-8bde-49b4-a75e-d592294df81d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:00.833552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:00.834430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:00.835267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:00.839643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 118/243] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:00.840475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38a76c-8bde-49b4-a75e-d592294df81d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:00.842283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216b43-f9c2-471e-9628-d776227c76f4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:00.889502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ee451-c316-4323-a358-62a0b49a7275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:00.890472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee451-c316-4323-a358-62a0b49a7275 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:00.891101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee451-c316-4323-a358-62a0b49a7275 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:00.891558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ee451-c316-4323-a358-62a0b49a7275 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:00.892831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ee451-c316-4323-a358-62a0b49a7275 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:00.894733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ee451-c316-4323-a358-62a0b49a7275 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:00.938835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:00.948995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38a76c-8bde-49b4-a75e-d592294df81d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:00.983023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:00.983636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:00.983917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:00.984188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:00.984853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:00.985157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:00.986551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:00.995339 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 119/244] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:00.996080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:01.009080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 126/245] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:55:00 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:01.010976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ee451-c316-4323-a358-62a0b49a7275 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:01.095455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:01.151741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ee451-c316-4323-a358-62a0b49a7275 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:01.168967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0a6c2-ab2b-431b-8d72-280b840168d7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:01.205438 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cda2f-9e86-4753-9ec9-756b67ed947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:01.205958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cda2f-9e86-4753-9ec9-756b67ed947c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.206238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cda2f-9e86-4753-9ec9-756b67ed947c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.206503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cda2f-9e86-4753-9ec9-756b67ed947c None admin] PATH_INFO: `/v3/users/975f839dc523488cbf97ab0c3b0ac096` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.212170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:01.213076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.213577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.214046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.214907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:01.215434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:01.217536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:01.222621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cda2f-9e86-4753-9ec9-756b67ed947c None admin] RBAC: Authorizing `identity:get_user(user_id=975f839dc523488cbf97ab0c3b0ac096)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:01.224245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cda2f-9e86-4753-9ec9-756b67ed947c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:01.227086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 127/246] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:01 2022] GET /identity/v3/users/975f839dc523488cbf97ab0c3b0ac096 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:01.227443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 120/247] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:01.232034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45cb5-ecfa-4033-a993-f915ed48db48 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.232034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45cb5-ecfa-4033-a993-f915ed48db48 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.232034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45cb5-ecfa-4033-a993-f915ed48db48 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.232034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 128/248] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:01.233088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:01.234253 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b45cb5-ecfa-4033-a993-f915ed48db48 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:01.253004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:01.360031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc42a7-727b-40b1-afb3-be301f454c23 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:01.375774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b45cb5-ecfa-4033-a993-f915ed48db48 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:01.398143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:01.414252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903abc2-20a9-4c43-933a-f6d482676023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:01.415246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903abc2-20a9-4c43-933a-f6d482676023 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.415841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903abc2-20a9-4c43-933a-f6d482676023 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.416312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903abc2-20a9-4c43-933a-f6d482676023 None admin] PATH_INFO: `/v3/users/20ead75f8301462fa4c8e66b4f142446` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.417669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903abc2-20a9-4c43-933a-f6d482676023 None admin] RBAC: Authorizing `identity:get_user(user_id=20ead75f8301462fa4c8e66b4f142446)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:01.420021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903abc2-20a9-4c43-933a-f6d482676023 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:01.422392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 129/249] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:01 2022] GET /identity/v3/users/20ead75f8301462fa4c8e66b4f142446 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:01.425584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764d9fe-a29e-4287-9f6b-5fafdf734fa3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.425584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764d9fe-a29e-4287-9f6b-5fafdf734fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.425584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764d9fe-a29e-4287-9f6b-5fafdf734fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.426867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed4b58-7aa0-4070-8ca1-cc860b05f07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:01.429024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4b58-7aa0-4070-8ca1-cc860b05f07b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.429024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4b58-7aa0-4070-8ca1-cc860b05f07b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.429024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4b58-7aa0-4070-8ca1-cc860b05f07b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.454481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d764d9fe-a29e-4287-9f6b-5fafdf734fa3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:01.455820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d764d9fe-a29e-4287-9f6b-5fafdf734fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:01.519100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4b58-7aa0-4070-8ca1-cc860b05f07b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:01.524871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4b58-7aa0-4070-8ca1-cc860b05f07b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:01.535440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 130/250] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:01.536305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764d9fe-a29e-4287-9f6b-5fafdf734fa3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:01.607338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:01.633539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 121/251] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:01.639264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4632c-d15f-4cc6-a9a6-8b8a8c99ebf9 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.639829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4632c-d15f-4cc6-a9a6-8b8a8c99ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.640284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4632c-d15f-4cc6-a9a6-8b8a8c99ebf9 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.641897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 122/252] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:01.642967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764d9fe-a29e-4287-9f6b-5fafdf734fa3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:01.644237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4632c-d15f-4cc6-a9a6-8b8a8c99ebf9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:01.661741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:01.681499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8301a94-2bc7-45a3-b54a-018c96114054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:01.682104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301a94-2bc7-45a3-b54a-018c96114054 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.682446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301a94-2bc7-45a3-b54a-018c96114054 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.682785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8301a94-2bc7-45a3-b54a-018c96114054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.756374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8301a94-2bc7-45a3-b54a-018c96114054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:01.757777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8301a94-2bc7-45a3-b54a-018c96114054 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:01.776261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4632c-d15f-4cc6-a9a6-8b8a8c99ebf9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:01.803299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:01.818133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f73cca-0c97-4847-9510-bda3949d4d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:01.822024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f73cca-0c97-4847-9510-bda3949d4d22 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.822024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f73cca-0c97-4847-9510-bda3949d4d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.822024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f73cca-0c97-4847-9510-bda3949d4d22 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.822024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f73cca-0c97-4847-9510-bda3949d4d22 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:01.823090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f73cca-0c97-4847-9510-bda3949d4d22 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:01.825013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 123/253] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:01 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:01.831536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25830cb4-6cea-4b36-9175-0c7e208a2598 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:01.832561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25830cb4-6cea-4b36-9175-0c7e208a2598 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:01.833080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25830cb4-6cea-4b36-9175-0c7e208a2598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:01.858246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-25830cb4-6cea-4b36-9175-0c7e208a2598 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:01.860330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-25830cb4-6cea-4b36-9175-0c7e208a2598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:01.868030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 131/254] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:01.869401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8301a94-2bc7-45a3-b54a-018c96114054 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:01.889541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:01.937099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 124/255] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:01.938402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25830cb4-6cea-4b36-9175-0c7e208a2598 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:02.002453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8301a94-2bc7-45a3-b54a-018c96114054 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:02.045604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25830cb4-6cea-4b36-9175-0c7e208a2598 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:02.047899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:02.049429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:02.049719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:02.049996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:02.050789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] RBAC: Adding query filter params (role.id=976d0bc0f2b14f26877bcc529d3645c7, scope.project.id=5749ebcaf01649d2a355e1191671ad21, user.id=975f839dc523488cbf97ab0c3b0ac096) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:02.051128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:02.052646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:02.065554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 132/256] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:01 2022] GET /identity/v3/role_assignments?user.id=975f839dc523488cbf97ab0c3b0ac096&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=976d0bc0f2b14f26877bcc529d3645c7 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:02.065554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:02.077503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:02.152227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d419a-b687-433c-b739-5f6fee6f4d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:02.152892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d419a-b687-433c-b739-5f6fee6f4d5a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:02.153191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d419a-b687-433c-b739-5f6fee6f4d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:02.153460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d419a-b687-433c-b739-5f6fee6f4d5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:02.160434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:02.194737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6377d0-f610-43ab-afc1-8ec668a6b8b2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:02.208005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d419a-b687-433c-b739-5f6fee6f4d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-044d419a-b687-433c-b739-5f6fee6f4d5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:02.212659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:02.213538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:02.214330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:02.215423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 125/257] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:02.220853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d419a-b687-433c-b739-5f6fee6f4d5a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:02.257450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12d480-cc6d-41d4-9326-6042912a8cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:02.258689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12d480-cc6d-41d4-9326-6042912a8cfd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:02.259219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12d480-cc6d-41d4-9326-6042912a8cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:02.259701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12d480-cc6d-41d4-9326-6042912a8cfd None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:02.261151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12d480-cc6d-41d4-9326-6042912a8cfd None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:02.263759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12d480-cc6d-41d4-9326-6042912a8cfd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:02.265820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 133/258] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:02 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:02.271462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12d480-cc6d-41d4-9326-6042912a8cfd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:02.431710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d419a-b687-433c-b739-5f6fee6f4d5a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:02.459602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12d480-cc6d-41d4-9326-6042912a8cfd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:02.497874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:02.498994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:02.499278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:02.499547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:02.500216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:02.500517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:02.502178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:02.512368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 126/259] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:02.518063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:02.556662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09019-a3fb-4c2f-97a1-944a4f0aa410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:02.556866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:02.557309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09019-a3fb-4c2f-97a1-944a4f0aa410 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:02.557587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09019-a3fb-4c2f-97a1-944a4f0aa410 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:02.557859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09019-a3fb-4c2f-97a1-944a4f0aa410 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:02.583869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a09019-a3fb-4c2f-97a1-944a4f0aa410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c8a09019-a3fb-4c2f-97a1-944a4f0aa410 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:02.588851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:02.589812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:02.590584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 134/260] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:55:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:02.596171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09019-a3fb-4c2f-97a1-944a4f0aa410 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:02.671210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:02.707378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34252d8d-2e5f-474a-b833-bebf1d19182a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:02.777516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09019-a3fb-4c2f-97a1-944a4f0aa410 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:02.795673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:02.796278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:02.796561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:02.796838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:02.797685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=5749ebcaf01649d2a355e1191671ad21, user.id=20ead75f8301462fa4c8e66b4f142446) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:02.797878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:02.799349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:02.818987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 127/261] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:02 2022] GET /identity/v3/role_assignments?user.id=20ead75f8301462fa4c8e66b4f142446&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:02.818987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:02.826986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537107c-96b7-445a-95dd-c4b04b171c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:02.828155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537107c-96b7-445a-95dd-c4b04b171c0e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:02.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537107c-96b7-445a-95dd-c4b04b171c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:02.829089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537107c-96b7-445a-95dd-c4b04b171c0e None admin] PATH_INFO: `/v3/users/493803e3313a47009f6071ad408a9a6e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:02.830720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537107c-96b7-445a-95dd-c4b04b171c0e None admin] RBAC: Authorizing `identity:get_user(user_id=493803e3313a47009f6071ad408a9a6e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:02.832575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537107c-96b7-445a-95dd-c4b04b171c0e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:02.834629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 135/262] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:02 2022] GET /identity/v3/users/493803e3313a47009f6071ad408a9a6e => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:02.840248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537107c-96b7-445a-95dd-c4b04b171c0e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:02.938197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:02.946118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97e4f2-ae9e-4a10-8ef1-54e23c41e6f3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:02.979464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:03.016767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537107c-96b7-445a-95dd-c4b04b171c0e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:03.029717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:03.030349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.030631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.030906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.031616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:03.031922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:03.033341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:03.050214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 128/263] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:03.056447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:03.088147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35221b-8f41-4f59-8347-dcad40d2eb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:03.088789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35221b-8f41-4f59-8347-dcad40d2eb57 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.089068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35221b-8f41-4f59-8347-dcad40d2eb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.089339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35221b-8f41-4f59-8347-dcad40d2eb57 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.090488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f35221b-8f41-4f59-8347-dcad40d2eb57 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:03.092048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f35221b-8f41-4f59-8347-dcad40d2eb57 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:03.093885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 136/264] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:02 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:03.100464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35221b-8f41-4f59-8347-dcad40d2eb57 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:03.213346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8d5b5-ee46-4b47-aeb9-625932442c74 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:03.258679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76dc8-82e1-42fd-bcff-e4e049126088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:03.259256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76dc8-82e1-42fd-bcff-e4e049126088 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.259541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76dc8-82e1-42fd-bcff-e4e049126088 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.259816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76dc8-82e1-42fd-bcff-e4e049126088 None admin] PATH_INFO: `/v3/users/20ead75f8301462fa4c8e66b4f142446` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.264180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d76dc8-82e1-42fd-bcff-e4e049126088 None admin] RBAC: Authorizing `identity:get_user(user_id=20ead75f8301462fa4c8e66b4f142446)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:03.265640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d76dc8-82e1-42fd-bcff-e4e049126088 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:03.269924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 129/265] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:03 2022] GET /identity/v3/users/20ead75f8301462fa4c8e66b4f142446 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:03.274310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76dc8-82e1-42fd-bcff-e4e049126088 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:03.292064 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35221b-8f41-4f59-8347-dcad40d2eb57 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:03.307327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:03.351649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26712ec-7f6b-4f71-84e5-a41fb54b7f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:03.352314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26712ec-7f6b-4f71-84e5-a41fb54b7f8d None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.352592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26712ec-7f6b-4f71-84e5-a41fb54b7f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.352857 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26712ec-7f6b-4f71-84e5-a41fb54b7f8d None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21/users/493803e3313a47009f6071ad408a9a6e/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.354826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26712ec-7f6b-4f71-84e5-a41fb54b7f8d None admin] RBAC: Authorizing `identity:create_grant(project_id=5749ebcaf01649d2a355e1191671ad21, user_id=493803e3313a47009f6071ad408a9a6e, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:03.356823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26712ec-7f6b-4f71-84e5-a41fb54b7f8d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:03.381958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76dc8-82e1-42fd-bcff-e4e049126088 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:03.417768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b4ca9-5425-4d6b-b0ed-f4647f776390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:03.418380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b4ca9-5425-4d6b-b0ed-f4647f776390 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.418664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b4ca9-5425-4d6b-b0ed-f4647f776390 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.419509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b4ca9-5425-4d6b-b0ed-f4647f776390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.436114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 137/266] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:55:03 2022] PUT /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21/users/493803e3313a47009f6071ad408a9a6e/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:03.496707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b4ca9-5425-4d6b-b0ed-f4647f776390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:03.498418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b4ca9-5425-4d6b-b0ed-f4647f776390 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:03.528745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:03.616119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 130/267] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:03.622437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26712ec-7f6b-4f71-84e5-a41fb54b7f8d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:03.632418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:03.666670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eefcf-355a-4ed2-a6af-f4ea8e29a9d9 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.667009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eefcf-355a-4ed2-a6af-f4ea8e29a9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.667333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703eefcf-355a-4ed2-a6af-f4ea8e29a9d9 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.669622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 131/268] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:55:03.676433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa33d6-2be2-44a5-8148-8f5acb19c3fb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.676714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa33d6-2be2-44a5-8148-8f5acb19c3fb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.676985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa33d6-2be2-44a5-8148-8f5acb19c3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.709841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5cfa33d6-2be2-44a5-8148-8f5acb19c3fb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:03.711047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5cfa33d6-2be2-44a5-8148-8f5acb19c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:03.807441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 132/269] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:03.808912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f9805-01fc-474c-8642-62bbd2f39bc3 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.809301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f9805-01fc-474c-8642-62bbd2f39bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.809711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f9805-01fc-474c-8642-62bbd2f39bc3 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.809919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26712ec-7f6b-4f71-84e5-a41fb54b7f8d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:03.811346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 133/270] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:03.816477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720ced-def5-444b-bae9-12f328cf0843 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.816864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720ced-def5-444b-bae9-12f328cf0843 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.817246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720ced-def5-444b-bae9-12f328cf0843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.876164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8f720ced-def5-444b-bae9-12f328cf0843 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:03.877379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8f720ced-def5-444b-bae9-12f328cf0843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:03.885495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9178ae-c5a5-45fd-a772-ad266db219f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:03.886571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9178ae-c5a5-45fd-a772-ad266db219f1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:03.886571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9178ae-c5a5-45fd-a772-ad266db219f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:03.886754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9178ae-c5a5-45fd-a772-ad266db219f1 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:03.892976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9178ae-c5a5-45fd-a772-ad266db219f1 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:03.896233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9178ae-c5a5-45fd-a772-ad266db219f1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:03.896233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 138/271] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:03 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:03.904705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9178ae-c5a5-45fd-a772-ad266db219f1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:03.996620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:04.037499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:04.053615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 134/272] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:04.100991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9178ae-c5a5-45fd-a772-ad266db219f1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:04.155501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f720ced-def5-444b-bae9-12f328cf0843 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:04.163358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:04.163962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:04.164239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:04.164506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:04.165231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] RBAC: Adding query filter params (role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=5749ebcaf01649d2a355e1191671ad21, user.id=20ead75f8301462fa4c8e66b4f142446) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:04.165580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:04.166990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:04.177133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 139/273] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:03 2022] GET /identity/v3/role_assignments?user.id=20ead75f8301462fa4c8e66b4f142446&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:04.382049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:04.398001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f720ced-def5-444b-bae9-12f328cf0843 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:04.459472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d0ddf-cdaa-47fc-873c-08ebe0c758aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:04.460876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d0ddf-cdaa-47fc-873c-08ebe0c758aa None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:04.460876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d0ddf-cdaa-47fc-873c-08ebe0c758aa None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:04.460876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d0ddf-cdaa-47fc-873c-08ebe0c758aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:04.461743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d0ddf-cdaa-47fc-873c-08ebe0c758aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:04.463441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d0ddf-cdaa-47fc-873c-08ebe0c758aa None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:04.465064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 135/274] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:04.471031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d0ddf-cdaa-47fc-873c-08ebe0c758aa None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:04.548222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d5167-ec8c-4c1c-8541-cc0ea90d7fec None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:04.563951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:04.597179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b6368-3134-4e96-8805-4126cdc13529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:04.599190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b6368-3134-4e96-8805-4126cdc13529 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:04.600500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b6368-3134-4e96-8805-4126cdc13529 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:04.602036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b6368-3134-4e96-8805-4126cdc13529 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:04.608477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:04.619279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b6368-3134-4e96-8805-4126cdc13529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3a0b6368-3134-4e96-8805-4126cdc13529 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:04.624116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:04.625188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:04.626219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:04.631024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 140/275] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:04.639608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bb5d1-30cd-48b0-8e79-e5dc3e386db8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:04.639608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bb5d1-30cd-48b0-8e79-e5dc3e386db8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:04.639608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bb5d1-30cd-48b0-8e79-e5dc3e386db8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:04.639608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 141/276] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:04.639608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bb5d1-30cd-48b0-8e79-e5dc3e386db8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:04.640107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d0ddf-cdaa-47fc-873c-08ebe0c758aa None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:04.673611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014777f9-40d0-48e5-8f46-a4cdd3a556dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:04.674314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014777f9-40d0-48e5-8f46-a4cdd3a556dd None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:04.674680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014777f9-40d0-48e5-8f46-a4cdd3a556dd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:04.675056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014777f9-40d0-48e5-8f46-a4cdd3a556dd None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:04.675925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014777f9-40d0-48e5-8f46-a4cdd3a556dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:04.678232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014777f9-40d0-48e5-8f46-a4cdd3a556dd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:04.742382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bb5d1-30cd-48b0-8e79-e5dc3e386db8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:04.767482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 136/277] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:55:04 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:04.770581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54f18-12cb-440f-9ba1-155c95274cfc None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:04.771096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54f18-12cb-440f-9ba1-155c95274cfc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:04.771509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54f18-12cb-440f-9ba1-155c95274cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:04.798970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:04.799182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-11a54f18-12cb-440f-9ba1-155c95274cfc None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:04.799260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:04.799260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:04.799260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:04.799464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-11a54f18-12cb-440f-9ba1-155c95274cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:04.799533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:04.799533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:04.799533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:04.813303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 142/278] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:04.819141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:04.880406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 137/279] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:04.924513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:04.945424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43182101-14c6-40c5-a588-9615ccad20f5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:04.983832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf68bbb-de12-4718-b709-5e61454f1f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:04.984689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf68bbb-de12-4718-b709-5e61454f1f62 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:04.985137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf68bbb-de12-4718-b709-5e61454f1f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:04.985552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf68bbb-de12-4718-b709-5e61454f1f62 None admin] PATH_INFO: `/v3/users/e6b39657940b4bd3adad068d0937719d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:05.000103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf68bbb-de12-4718-b709-5e61454f1f62 None admin] RBAC: Authorizing `identity:get_user(user_id=e6b39657940b4bd3adad068d0937719d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:05.001769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf68bbb-de12-4718-b709-5e61454f1f62 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:05.003892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 143/280] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:04 2022] GET /identity/v3/users/e6b39657940b4bd3adad068d0937719d => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:05.009657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a54f18-12cb-440f-9ba1-155c95274cfc None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:05.056526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:05.115513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a54f18-12cb-440f-9ba1-155c95274cfc None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:05.155102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf68bbb-de12-4718-b709-5e61454f1f62 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:05.158845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:05.159490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:05.159764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:05.160030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:05.160783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] RBAC: Adding query filter params (role.id=3b325dcac3064617a5415a84759ef71a, user.id=e6b39657940b4bd3adad068d0937719d, scope.system=all) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:05.161067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:05.162640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:05.171274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 138/281] 10.222.0.58 () {56 vars in 1308 bytes} [Tue Aug 23 07:55:05 2022] GET /identity/v3/role_assignments?user.id=e6b39657940b4bd3adad068d0937719d&scope.system=all&role.id=3b325dcac3064617a5415a84759ef71a => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:05.281148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf68bbb-de12-4718-b709-5e61454f1f62 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:05.301048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:05.321008 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f801-d781-49ea-b358-d1a1406a9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:05.321641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f801-d781-49ea-b358-d1a1406a9e95 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:05.322175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f801-d781-49ea-b358-d1a1406a9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:05.322449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f801-d781-49ea-b358-d1a1406a9e95 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:05.334826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465f801-d781-49ea-b358-d1a1406a9e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6465f801-d781-49ea-b358-d1a1406a9e95 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:05.339600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:05.340473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:05.341335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 144/282] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:05.352050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:05.504149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec22efb-bf2f-4222-af10-b2cd26b09936 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:05.517911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:05.555805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:05.556513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:05.557116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:05.557387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:05.558056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:05.558355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:05.559803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:05.570236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 139/283] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:05.576764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f801-d781-49ea-b358-d1a1406a9e95 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:05.743121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f801-d781-49ea-b358-d1a1406a9e95 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:05.799663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cf611-c14d-476d-988e-db69f7c51909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:05.800298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf611-c14d-476d-988e-db69f7c51909 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:05.800579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf611-c14d-476d-988e-db69f7c51909 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:05.800850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf611-c14d-476d-988e-db69f7c51909 None admin] PATH_INFO: `/v3/users/975f839dc523488cbf97ab0c3b0ac096` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:05.802023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cf611-c14d-476d-988e-db69f7c51909 None admin] RBAC: Authorizing `identity:get_user(user_id=975f839dc523488cbf97ab0c3b0ac096)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:05.803496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cf611-c14d-476d-988e-db69f7c51909 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:05.805072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 145/284] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:05 2022] GET /identity/v3/users/975f839dc523488cbf97ab0c3b0ac096 => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:05.811569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:05.827582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6370f-0cb0-4a39-8058-59dea1a9df7f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:05.827853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6370f-0cb0-4a39-8058-59dea1a9df7f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:05.828123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6370f-0cb0-4a39-8058-59dea1a9df7f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:05.831035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 146/285] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:05.834878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a03c7-6649-44f8-b8d5-416f62ef6a33 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:05.835182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a03c7-6649-44f8-b8d5-416f62ef6a33 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:05.835451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a03c7-6649-44f8-b8d5-416f62ef6a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:05.865769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-256a03c7-6649-44f8-b8d5-416f62ef6a33 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:05.866958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-256a03c7-6649-44f8-b8d5-416f62ef6a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:05.944173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:05.983243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc0b3c-a49b-4866-ac2d-587dc5c97d6d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:05.991725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 147/286] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:06.052174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a291e40-ce8b-4e1b-bb35-86f24ecb3bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:06.053407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a291e40-ce8b-4e1b-bb35-86f24ecb3bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:06.053832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a291e40-ce8b-4e1b-bb35-86f24ecb3bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:06.054323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a291e40-ce8b-4e1b-bb35-86f24ecb3bc8 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:06.055628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a291e40-ce8b-4e1b-bb35-86f24ecb3bc8 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:06.057239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a291e40-ce8b-4e1b-bb35-86f24ecb3bc8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:06.059025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 140/287] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:05 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:06.065195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a03c7-6649-44f8-b8d5-416f62ef6a33 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:06.256972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a03c7-6649-44f8-b8d5-416f62ef6a33 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:06.324004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a291e40-ce8b-4e1b-bb35-86f24ecb3bc8 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:06.329630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943abf9-bf8b-4ece-b32c-67e74aa18126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:06.330304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943abf9-bf8b-4ece-b32c-67e74aa18126 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:06.330583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943abf9-bf8b-4ece-b32c-67e74aa18126 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:06.330857 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943abf9-bf8b-4ece-b32c-67e74aa18126 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21/users/975f839dc523488cbf97ab0c3b0ac096/roles/976d0bc0f2b14f26877bcc529d3645c7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:06.333017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943abf9-bf8b-4ece-b32c-67e74aa18126 None admin] RBAC: Authorizing `identity:create_grant(project_id=5749ebcaf01649d2a355e1191671ad21, user_id=975f839dc523488cbf97ab0c3b0ac096, role_id=976d0bc0f2b14f26877bcc529d3645c7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:06.334803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943abf9-bf8b-4ece-b32c-67e74aa18126 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:06.398975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:06.418380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 148/288] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:55:06 2022] PUT /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21/users/975f839dc523488cbf97ab0c3b0ac096/roles/976d0bc0f2b14f26877bcc529d3645c7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:06.557209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a291e40-ce8b-4e1b-bb35-86f24ecb3bc8 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:06.629567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3d360-94a6-4aef-b13d-81d081a01b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:06.630126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d360-94a6-4aef-b13d-81d081a01b81 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:06.630409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d360-94a6-4aef-b13d-81d081a01b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:06.630747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3d360-94a6-4aef-b13d-81d081a01b81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:06.660982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3d360-94a6-4aef-b13d-81d081a01b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cae3d360-94a6-4aef-b13d-81d081a01b81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:06.671656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:06.672551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:06.673338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 141/289] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:06.681167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943abf9-bf8b-4ece-b32c-67e74aa18126 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:06.750994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:06.852727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943abf9-bf8b-4ece-b32c-67e74aa18126 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:06.889693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:06.890229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:06.890508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:06.890779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:06.891475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:06.891773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:06.893142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:06.901923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 149/290] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:06.907404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3d360-94a6-4aef-b13d-81d081a01b81 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:07.035638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:07.041967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3d360-94a6-4aef-b13d-81d081a01b81 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:07.095706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76001d4-251a-4a00-bb9f-96e61828c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:07.096325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76001d4-251a-4a00-bb9f-96e61828c204 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.096611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76001d4-251a-4a00-bb9f-96e61828c204 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.096883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76001d4-251a-4a00-bb9f-96e61828c204 None admin] PATH_INFO: `/v3/users/493803e3313a47009f6071ad408a9a6e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.103625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76001d4-251a-4a00-bb9f-96e61828c204 None admin] RBAC: Authorizing `identity:get_user(user_id=493803e3313a47009f6071ad408a9a6e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:07.105101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76001d4-251a-4a00-bb9f-96e61828c204 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:07.107477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 142/291] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:06 2022] GET /identity/v3/users/493803e3313a47009f6071ad408a9a6e => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:07.113092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:07.248814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:07.259680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0320-ce76-4204-a139-9e7cb7bebf05 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:07.295983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb786de-dec3-4de8-ae4d-0d7daf066316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:07.296595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb786de-dec3-4de8-ae4d-0d7daf066316 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.296877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb786de-dec3-4de8-ae4d-0d7daf066316 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.297152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb786de-dec3-4de8-ae4d-0d7daf066316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.367144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb786de-dec3-4de8-ae4d-0d7daf066316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:07.368610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb786de-dec3-4de8-ae4d-0d7daf066316 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:07.464620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:07.471976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 150/292] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:07.477709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76001d4-251a-4a00-bb9f-96e61828c204 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:07.531736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea2b8f-2571-46d3-b310-95333f729cfa None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.532356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea2b8f-2571-46d3-b310-95333f729cfa None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.532822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea2b8f-2571-46d3-b310-95333f729cfa None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.534368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 151/293] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:55:07.540207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3ca0-11f5-40c0-8bd4-a03d4de3e3f5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.540736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3ca0-11f5-40c0-8bd4-a03d4de3e3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.541642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b3ca0-11f5-40c0-8bd4-a03d4de3e3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.562407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c35b3ca0-11f5-40c0-8bd4-a03d4de3e3f5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:07.564084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c35b3ca0-11f5-40c0-8bd4-a03d4de3e3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:07.621695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76001d4-251a-4a00-bb9f-96e61828c204 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:07.654310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 152/294] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:07.654310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a48e8-d2dd-452f-a53d-24f4217e6472 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.654310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a48e8-d2dd-452f-a53d-24f4217e6472 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.654310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a48e8-d2dd-452f-a53d-24f4217e6472 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.654682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 153/295] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:07.663127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b48ea-ff6c-43dd-8bb0-ef96b21018b2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.665214 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b48ea-ff6c-43dd-8bb0-ef96b21018b2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.665581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b48ea-ff6c-43dd-8bb0-ef96b21018b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.692387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577ee16-799c-4533-989a-ba8480f9352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:07.693465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577ee16-799c-4533-989a-ba8480f9352c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.693759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577ee16-799c-4533-989a-ba8480f9352c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.694036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577ee16-799c-4533-989a-ba8480f9352c None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.695426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577ee16-799c-4533-989a-ba8480f9352c None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:07.697224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577ee16-799c-4533-989a-ba8480f9352c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:07.698695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bf4b48ea-ff6c-43dd-8bb0-ef96b21018b2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:07.699242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 143/296] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:07 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:07.699982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bf4b48ea-ff6c-43dd-8bb0-ef96b21018b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:07.700410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011601c-87c4-4151-8756-cb00b297825e None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.700692 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011601c-87c4-4151-8756-cb00b297825e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.700966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011601c-87c4-4151-8756-cb00b297825e None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.706466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 144/297] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:07.708804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad1039-48ca-4a3f-9630-4e61660a9ef1 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:07.709102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad1039-48ca-4a3f-9630-4e61660a9ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:07.709384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad1039-48ca-4a3f-9630-4e61660a9ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:07.730434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3cad1039-48ca-4a3f-9630-4e61660a9ef1 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:07.732867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3cad1039-48ca-4a3f-9630-4e61660a9ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:07.771961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:07.785675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 154/298] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:07.790973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b48ea-ff6c-43dd-8bb0-ef96b21018b2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:07.822979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 145/299] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:07.894411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:07.945852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b48ea-ff6c-43dd-8bb0-ef96b21018b2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:07.970086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad1039-48ca-4a3f-9630-4e61660a9ef1 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.007378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.007990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.008265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.008536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.009268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] RBAC: Adding query filter params (role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=5749ebcaf01649d2a355e1191671ad21, user.id=493803e3313a47009f6071ad408a9a6e) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:08.009573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:08.011047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:08.024662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 155/300] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:07 2022] GET /identity/v3/role_assignments?user.id=493803e3313a47009f6071ad408a9a6e&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=3b325dcac3064617a5415a84759ef71a => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:08.100340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad1039-48ca-4a3f-9630-4e61660a9ef1 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.126591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.155855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6f32d-af2c-4728-b3e0-97d993d720af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.156858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6f32d-af2c-4728-b3e0-97d993d720af None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.157398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6f32d-af2c-4728-b3e0-97d993d720af None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.157866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6f32d-af2c-4728-b3e0-97d993d720af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.178132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6f32d-af2c-4728-b3e0-97d993d720af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-4eb6f32d-af2c-4728-b3e0-97d993d720af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:08.184770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:08.185846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:08.186979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 146/301] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:08.190800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6f32d-af2c-4728-b3e0-97d993d720af None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.226117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:08.273927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb4406-3d27-4ed7-8a05-7161d2d8b8dd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.296518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:08.313221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e0cea-7b17-4546-aec7-36c529552f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.314335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e0cea-7b17-4546-aec7-36c529552f4c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.315346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e0cea-7b17-4546-aec7-36c529552f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.315629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e0cea-7b17-4546-aec7-36c529552f4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.316823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e0cea-7b17-4546-aec7-36c529552f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:08.316945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6f32d-af2c-4728-b3e0-97d993d720af None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.318310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e0cea-7b17-4546-aec7-36c529552f4c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:08.322988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 156/302] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:08.322988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e0cea-7b17-4546-aec7-36c529552f4c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.371199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.371760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.372105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.372615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.373026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:08.373320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:08.374705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:08.386026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 147/303] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:08.386858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.457983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e0cea-7b17-4546-aec7-36c529552f4c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.479840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:08.493858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91bc04-eb67-4134-8a67-eb75856bd683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.494486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91bc04-eb67-4134-8a67-eb75856bd683 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.494764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91bc04-eb67-4134-8a67-eb75856bd683 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.495066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91bc04-eb67-4134-8a67-eb75856bd683 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.497453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ff504-e4d1-4daf-9004-7b1f77ff293b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.505273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91bc04-eb67-4134-8a67-eb75856bd683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5d91bc04-eb67-4134-8a67-eb75856bd683 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:08.510424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:08.511316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:08.512143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 157/304] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:08.513545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91bc04-eb67-4134-8a67-eb75856bd683 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.535860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24961c2-6285-4065-b025-ccc9fd13bd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.536471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24961c2-6285-4065-b025-ccc9fd13bd94 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.536746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24961c2-6285-4065-b025-ccc9fd13bd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.537018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24961c2-6285-4065-b025-ccc9fd13bd94 None admin] PATH_INFO: `/v3/groups` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.537635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24961c2-6285-4065-b025-ccc9fd13bd94 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:08.539099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24961c2-6285-4065-b025-ccc9fd13bd94 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:08.566335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:08.615685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91bc04-eb67-4134-8a67-eb75856bd683 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.648604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc6134-6b40-4e98-a704-c24c7ba8455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.649131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc6134-6b40-4e98-a704-c24c7ba8455c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.649403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc6134-6b40-4e98-a704-c24c7ba8455c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.649671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc6134-6b40-4e98-a704-c24c7ba8455c None admin] PATH_INFO: `/v3/users/e6b39657940b4bd3adad068d0937719d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.650780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc6134-6b40-4e98-a704-c24c7ba8455c None admin] RBAC: Authorizing `identity:get_user(user_id=e6b39657940b4bd3adad068d0937719d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:08.652242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc6134-6b40-4e98-a704-c24c7ba8455c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:08.654485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 158/305] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:08 2022] GET /identity/v3/users/e6b39657940b4bd3adad068d0937719d => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:08.655172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc6134-6b40-4e98-a704-c24c7ba8455c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.661843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 148/306] 10.222.0.58 () {60 vars in 1134 bytes} [Tue Aug 23 07:55:08 2022] POST /identity/v3/groups => generated 229 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:08.663064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24961c2-6285-4065-b025-ccc9fd13bd94 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.796985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc6134-6b40-4e98-a704-c24c7ba8455c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.812083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24961c2-6285-4065-b025-ccc9fd13bd94 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.817814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:08.839729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb76e0-9716-4319-853b-33cfaa76b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.840342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb76e0-9716-4319-853b-33cfaa76b2ef None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.840654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb76e0-9716-4319-853b-33cfaa76b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.840980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb76e0-9716-4319-853b-33cfaa76b2ef None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.841655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb76e0-9716-4319-853b-33cfaa76b2ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:08.841961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb76e0-9716-4319-853b-33cfaa76b2ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:08.843438 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb76e0-9716-4319-853b-33cfaa76b2ef None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:08.852114 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6398ad-c84b-45c4-ba2e-b72f51ceaf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:08.852768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6398ad-c84b-45c4-ba2e-b72f51ceaf4d None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.853045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6398ad-c84b-45c4-ba2e-b72f51ceaf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.853519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6398ad-c84b-45c4-ba2e-b72f51ceaf4d None admin] PATH_INFO: `/v3/system/users/e6b39657940b4bd3adad068d0937719d/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.853623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 159/307] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:08.854570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda242e-4636-4cab-91f1-6f3bd98d2eca None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.854853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda242e-4636-4cab-91f1-6f3bd98d2eca None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.855165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda242e-4636-4cab-91f1-6f3bd98d2eca None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.855950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6398ad-c84b-45c4-ba2e-b72f51ceaf4d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e6b39657940b4bd3adad068d0937719d, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:08.857499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 160/308] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:08.857876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6398ad-c84b-45c4-ba2e-b72f51ceaf4d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:08.859629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda242e-4636-4cab-91f1-6f3bd98d2eca None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:08.945955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 149/309] 10.222.0.58 () {58 vars in 1334 bytes} [Tue Aug 23 07:55:08 2022] PUT /identity/v3/system/users/e6b39657940b4bd3adad068d0937719d/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:08.947553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955dcc4-5ff9-4011-a1db-9470bb7e67ba None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:08.947842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955dcc4-5ff9-4011-a1db-9470bb7e67ba None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:08.948112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955dcc4-5ff9-4011-a1db-9470bb7e67ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:08.968507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d955dcc4-5ff9-4011-a1db-9470bb7e67ba None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:08.969844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d955dcc4-5ff9-4011-a1db-9470bb7e67ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:08.972747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda242e-4636-4cab-91f1-6f3bd98d2eca None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:08.979385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:09.013119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa3985-eb0f-483c-a4a7-482874f69e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:09.013903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa3985-eb0f-483c-a4a7-482874f69e82 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:09.014192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa3985-eb0f-483c-a4a7-482874f69e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:09.014457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa3985-eb0f-483c-a4a7-482874f69e82 None admin] PATH_INFO: `/v3/users/7848f7e8434540858529cdd31e7a68d9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:09.030667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa3985-eb0f-483c-a4a7-482874f69e82 None admin] RBAC: Authorizing `identity:get_user(user_id=7848f7e8434540858529cdd31e7a68d9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:09.032339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa3985-eb0f-483c-a4a7-482874f69e82 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:09.038668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 161/310] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:08 2022] GET /identity/v3/users/7848f7e8434540858529cdd31e7a68d9 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:09.040247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa3985-eb0f-483c-a4a7-482874f69e82 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:09.065710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 150/311] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:09.081911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:09.152368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa3985-eb0f-483c-a4a7-482874f69e82 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:09.189700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:09.190543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:09.190990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:09.191404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:09.192295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] RBAC: Adding query filter params (role.id=976d0bc0f2b14f26877bcc529d3645c7, user.id=7848f7e8434540858529cdd31e7a68d9, scope.system=all) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:09.192749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:09.194334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:09.200569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 162/312] 10.222.0.58 () {56 vars in 1308 bytes} [Tue Aug 23 07:55:09 2022] GET /identity/v3/role_assignments?user.id=7848f7e8434540858529cdd31e7a68d9&scope.system=all&role.id=976d0bc0f2b14f26877bcc529d3645c7 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:09.338331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955dcc4-5ff9-4011-a1db-9470bb7e67ba None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:09.445497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955dcc4-5ff9-4011-a1db-9470bb7e67ba None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:09.481203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef0d2-f5ed-4405-b137-9ae38154484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:09.482175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef0d2-f5ed-4405-b137-9ae38154484d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:09.482453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef0d2-f5ed-4405-b137-9ae38154484d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:09.482721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ef0d2-f5ed-4405-b137-9ae38154484d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:09.489916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:09.493603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ef0d2-f5ed-4405-b137-9ae38154484d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-737ef0d2-f5ed-4405-b137-9ae38154484d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:09.498561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:09.511337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:09.512123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 151/313] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:09.512859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:09.615466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:09.618843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e195c-91fa-4e4b-a966-3a46381cd8a9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:09.655309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:09.655917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:09.656250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:09.656525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:09.657187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:09.657489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:09.658859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:09.667743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 163/314] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:09.674024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef0d2-f5ed-4405-b137-9ae38154484d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:09.783229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ef0d2-f5ed-4405-b137-9ae38154484d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:09.819861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb84bb1-277b-416d-b309-6bd70d24b516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:09.820497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb84bb1-277b-416d-b309-6bd70d24b516 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:09.820769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb84bb1-277b-416d-b309-6bd70d24b516 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:09.821034 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb84bb1-277b-416d-b309-6bd70d24b516 None admin] PATH_INFO: `/v3/users/975f839dc523488cbf97ab0c3b0ac096` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:09.822422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb84bb1-277b-416d-b309-6bd70d24b516 None admin] RBAC: Authorizing `identity:get_user(user_id=975f839dc523488cbf97ab0c3b0ac096)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:09.823888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb84bb1-277b-416d-b309-6bd70d24b516 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:09.826326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 152/315] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:09 2022] GET /identity/v3/users/975f839dc523488cbf97ab0c3b0ac096 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:09.832011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:09.939670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0b41c-b733-42e5-b2e1-1450edc9f937 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:09.974814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b54487-d252-4c4c-b0a4-c0833e333b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:09.975454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b54487-d252-4c4c-b0a4-c0833e333b9f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:09.975733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b54487-d252-4c4c-b0a4-c0833e333b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:09.976005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b54487-d252-4c4c-b0a4-c0833e333b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:10.010806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:10.049946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b54487-d252-4c4c-b0a4-c0833e333b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:10.051345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b54487-d252-4c4c-b0a4-c0833e333b9f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:10.153900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 164/316] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:10.159696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb84bb1-277b-416d-b309-6bd70d24b516 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:10.230164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:10.264753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb84bb1-277b-416d-b309-6bd70d24b516 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:10.300936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaab615-eb05-4298-bdd8-051c2ffb99bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:10.301498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab615-eb05-4298-bdd8-051c2ffb99bd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:10.301775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab615-eb05-4298-bdd8-051c2ffb99bd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:10.302046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab615-eb05-4298-bdd8-051c2ffb99bd None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:10.303271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaab615-eb05-4298-bdd8-051c2ffb99bd None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:10.304752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaab615-eb05-4298-bdd8-051c2ffb99bd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:10.306916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 153/317] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:10 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:10.313303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b54487-d252-4c4c-b0a4-c0833e333b9f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:10.383534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:10.414300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b54487-d252-4c4c-b0a4-c0833e333b9f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:10.449175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d7023-e672-4870-9df0-8eaeb2f03fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:10.450155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d7023-e672-4870-9df0-8eaeb2f03fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:10.450634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d7023-e672-4870-9df0-8eaeb2f03fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:10.451169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d7023-e672-4870-9df0-8eaeb2f03fc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:10.452087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d7023-e672-4870-9df0-8eaeb2f03fc5 None admin] RBAC: Adding query filter params (role.id=976d0bc0f2b14f26877bcc529d3645c7, scope.project.id=5749ebcaf01649d2a355e1191671ad21, user.id=975f839dc523488cbf97ab0c3b0ac096) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:10.452595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d7023-e672-4870-9df0-8eaeb2f03fc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:10.454294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d7023-e672-4870-9df0-8eaeb2f03fc5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:10.465215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 165/318] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:10 2022] GET /identity/v3/role_assignments?user.id=975f839dc523488cbf97ab0c3b0ac096&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=976d0bc0f2b14f26877bcc529d3645c7 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:10.693180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e782c6-6709-4668-9eb8-d280dc94abc0 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:10.693675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e782c6-6709-4668-9eb8-d280dc94abc0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:10.693749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e782c6-6709-4668-9eb8-d280dc94abc0 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:10.695366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 154/319] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:10.701060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32f0e-6cb9-4676-8781-6939bfe6d127 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:10.701456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32f0e-6cb9-4676-8781-6939bfe6d127 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:10.701625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32f0e-6cb9-4676-8781-6939bfe6d127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:10.720069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-90b32f0e-6cb9-4676-8781-6939bfe6d127 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:10.721260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-90b32f0e-6cb9-4676-8781-6939bfe6d127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:10.794806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 166/320] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:10.973212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e046a3-d880-42ec-b372-0c1b6c0c3559 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:10.973495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e046a3-d880-42ec-b372-0c1b6c0c3559 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:10.973687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e046a3-d880-42ec-b372-0c1b6c0c3559 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:10.975211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 155/321] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:10.980174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc9444-15cc-41b7-a133-ba424f9e3641 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:10.980467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc9444-15cc-41b7-a133-ba424f9e3641 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:10.980750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc9444-15cc-41b7-a133-ba424f9e3641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:10.999343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-30fc9444-15cc-41b7-a133-ba424f9e3641 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:11.000647 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-30fc9444-15cc-41b7-a133-ba424f9e3641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:11.052848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:11.060780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e046a3-d880-42ec-b372-0c1b6c0c3559 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:11.070568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 167/322] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:11.136705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701f1a-3973-4b24-98fb-29a7f632771d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.137016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701f1a-3973-4b24-98fb-29a7f632771d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.137283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701f1a-3973-4b24-98fb-29a7f632771d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.139196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 168/323] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:55:11.143955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096955e0-dc64-4d96-bffb-aed1e83667a2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.144233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096955e0-dc64-4d96-bffb-aed1e83667a2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.144503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096955e0-dc64-4d96-bffb-aed1e83667a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.161909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-096955e0-dc64-4d96-bffb-aed1e83667a2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:11.163129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-096955e0-dc64-4d96-bffb-aed1e83667a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:11.164930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e046a3-d880-42ec-b372-0c1b6c0c3559 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:11.199963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496eae7e-43c1-4828-bb9c-f4ed1e905c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:11.200575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eae7e-43c1-4828-bb9c-f4ed1e905c93 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.200848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eae7e-43c1-4828-bb9c-f4ed1e905c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.201117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eae7e-43c1-4828-bb9c-f4ed1e905c93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.202198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496eae7e-43c1-4828-bb9c-f4ed1e905c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:11.203674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496eae7e-43c1-4828-bb9c-f4ed1e905c93 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:11.206986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 156/324] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:11.210622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496eae7e-43c1-4828-bb9c-f4ed1e905c93 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:11.213211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:11.235550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 169/325] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:11.309898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496eae7e-43c1-4828-bb9c-f4ed1e905c93 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:11.337116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096955e0-dc64-4d96-bffb-aed1e83667a2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:11.342828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c97f8-9c6d-4724-bce7-0f4b20931c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:11.343464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c97f8-9c6d-4724-bce7-0f4b20931c4d None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.343740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c97f8-9c6d-4724-bce7-0f4b20931c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.344005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c97f8-9c6d-4724-bce7-0f4b20931c4d None admin] PATH_INFO: `/v3/groups` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.344612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c97f8-9c6d-4724-bce7-0f4b20931c4d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:11.346045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c97f8-9c6d-4724-bce7-0f4b20931c4d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:11.434899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 157/326] 10.222.0.58 () {60 vars in 1134 bytes} [Tue Aug 23 07:55:11 2022] POST /identity/v3/groups => generated 226 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:55:11.438075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096955e0-dc64-4d96-bffb-aed1e83667a2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:11.457724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:11.477315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c6db-d8d7-42ed-b96d-cd450aa7f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:11.477961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c6db-d8d7-42ed-b96d-cd450aa7f2ab None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.478262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c6db-d8d7-42ed-b96d-cd450aa7f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.478536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88c6db-d8d7-42ed-b96d-cd450aa7f2ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.491273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88c6db-d8d7-42ed-b96d-cd450aa7f2ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4c88c6db-d8d7-42ed-b96d-cd450aa7f2ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:11.496066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:11.497632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:11.498405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:11.501839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c97f8-9c6d-4724-bce7-0f4b20931c4d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:11.502748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 170/327] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:11.503560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c6db-d8d7-42ed-b96d-cd450aa7f2ab None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:11.571309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:11.616236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88c6db-d8d7-42ed-b96d-cd450aa7f2ab None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:11.621243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c97f8-9c6d-4724-bce7-0f4b20931c4d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:11.656462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30e590-d2f5-415c-a03e-4207bb010396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:11.657272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:11.657272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.657434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30e590-d2f5-415c-a03e-4207bb010396 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.657503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.657597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30e590-d2f5-415c-a03e-4207bb010396 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.657668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.657734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30e590-d2f5-415c-a03e-4207bb010396 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.658247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:11.658547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:11.659975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:11.670182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 171/328] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:11.672326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30e590-d2f5-415c-a03e-4207bb010396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:11.676490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-ef30e590-d2f5-415c-a03e-4207bb010396 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:11.677665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:11.678579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:11.679392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 158/329] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:11.685094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30e590-d2f5-415c-a03e-4207bb010396 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:11.725536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:11.815455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7647702-597b-497c-bf47-68d8ac13fd48 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:11.823697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30e590-d2f5-415c-a03e-4207bb010396 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:11.851561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:11.857923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18486-470a-434e-bae2-3e33a19908dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:11.858931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18486-470a-434e-bae2-3e33a19908dd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.859717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18486-470a-434e-bae2-3e33a19908dd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.860138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18486-470a-434e-bae2-3e33a19908dd None admin] PATH_INFO: `/v3/users/e6b39657940b4bd3adad068d0937719d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.862078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18486-470a-434e-bae2-3e33a19908dd None admin] RBAC: Authorizing `identity:get_user(user_id=e6b39657940b4bd3adad068d0937719d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:11.862991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18486-470a-434e-bae2-3e33a19908dd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:11.864854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 172/330] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:11 2022] GET /identity/v3/users/e6b39657940b4bd3adad068d0937719d => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:11.867678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:11.868292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:11.868568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:11.868988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:11.869663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:11.870040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:11.870995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18486-470a-434e-bae2-3e33a19908dd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:11.873309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:11.883058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 159/331] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:11.890355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:12.004931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18486-470a-434e-bae2-3e33a19908dd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:12.013727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:12.016028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45594279-e4ff-426a-9ed7-de181574dfdf None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:12.045503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:12.046652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:12.047180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:12.047689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:12.048702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] RBAC: Adding query filter params (role.id=3b325dcac3064617a5415a84759ef71a, user.id=e6b39657940b4bd3adad068d0937719d, scope.system=all) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:12.049216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:12.050855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:12.061988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193300b-1fd7-4f74-bbf4-b82b3c3b504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:12.062628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193300b-1fd7-4f74-bbf4-b82b3c3b504e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:12.062923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193300b-1fd7-4f74-bbf4-b82b3c3b504e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:12.063302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193300b-1fd7-4f74-bbf4-b82b3c3b504e None admin] PATH_INFO: `/v3/users/7848f7e8434540858529cdd31e7a68d9` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:12.064739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193300b-1fd7-4f74-bbf4-b82b3c3b504e None admin] RBAC: Authorizing `identity:get_user(user_id=7848f7e8434540858529cdd31e7a68d9)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:12.066205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193300b-1fd7-4f74-bbf4-b82b3c3b504e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:12.066782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 173/332] 10.222.0.58 () {56 vars in 1308 bytes} [Tue Aug 23 07:55:11 2022] GET /identity/v3/role_assignments?user.id=e6b39657940b4bd3adad068d0937719d&scope.system=all&role.id=3b325dcac3064617a5415a84759ef71a => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:12.067948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 160/333] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:11 2022] GET /identity/v3/users/7848f7e8434540858529cdd31e7a68d9 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:12.076167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:12.189342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f3498-97e9-4a79-a80e-44715c63b56e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:12.227638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca0e4-a2fe-4d60-8c5f-f30ba834fc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:12.228302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca0e4-a2fe-4d60-8c5f-f30ba834fc7d None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:12.228584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca0e4-a2fe-4d60-8c5f-f30ba834fc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:12.228861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca0e4-a2fe-4d60-8c5f-f30ba834fc7d None admin] PATH_INFO: `/v3/system/users/7848f7e8434540858529cdd31e7a68d9/roles/976d0bc0f2b14f26877bcc529d3645c7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:12.230321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aca0e4-a2fe-4d60-8c5f-f30ba834fc7d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7848f7e8434540858529cdd31e7a68d9, role_id=976d0bc0f2b14f26877bcc529d3645c7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:12.232257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aca0e4-a2fe-4d60-8c5f-f30ba834fc7d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:12.311459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 174/334] 10.222.0.58 () {58 vars in 1334 bytes} [Tue Aug 23 07:55:12 2022] PUT /identity/v3/system/users/7848f7e8434540858529cdd31e7a68d9/roles/976d0bc0f2b14f26877bcc529d3645c7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:13.752856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64283-045a-4a34-b9af-5872b6cbd3e6 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:13.753371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64283-045a-4a34-b9af-5872b6cbd3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:13.753641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64283-045a-4a34-b9af-5872b6cbd3e6 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:13.755189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 161/335] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:13.756660 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7480-33ce-436a-bce7-11e17503624c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:13.757044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7480-33ce-436a-bce7-11e17503624c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:13.757221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7480-33ce-436a-bce7-11e17503624c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:13.758513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 175/336] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:13.764907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35a71d-63eb-47e5-85ec-a78152134b56 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:13.765186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35a71d-63eb-47e5-85ec-a78152134b56 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:13.765455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35a71d-63eb-47e5-85ec-a78152134b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:13.781020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1dbd-624e-4f48-af8b-1133fcd1a3b0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:13.781314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1dbd-624e-4f48-af8b-1133fcd1a3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:13.781561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1dbd-624e-4f48-af8b-1133fcd1a3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:13.798860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-be35a71d-63eb-47e5-85ec-a78152134b56 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:13.800094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-be35a71d-63eb-47e5-85ec-a78152134b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:13.827462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cbca1dbd-624e-4f48-af8b-1133fcd1a3b0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:13.828653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cbca1dbd-624e-4f48-af8b-1133fcd1a3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:13.873490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:13.894214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:13.933442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 162/337] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:13.964136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 176/338] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:14.297130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35a71d-63eb-47e5-85ec-a78152134b56 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:14.321386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1dbd-624e-4f48-af8b-1133fcd1a3b0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:14.486812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35a71d-63eb-47e5-85ec-a78152134b56 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:14.528128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1dbd-624e-4f48-af8b-1133fcd1a3b0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:14.544491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1eba7-6149-4a84-9c6c-cc0e6774f013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:14.545744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1eba7-6149-4a84-9c6c-cc0e6774f013 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:14.553944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1eba7-6149-4a84-9c6c-cc0e6774f013 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:14.553944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1eba7-6149-4a84-9c6c-cc0e6774f013 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:14.563551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1eba7-6149-4a84-9c6c-cc0e6774f013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-70d1eba7-6149-4a84-9c6c-cc0e6774f013 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:14.569879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:14.570794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:14.571634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:14.578976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 163/339] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:55:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:14.581166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1eba7-6149-4a84-9c6c-cc0e6774f013 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:14.586191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad635e0-323c-43e4-897d-20c352cba1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:14.586191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad635e0-323c-43e4-897d-20c352cba1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:14.586191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad635e0-323c-43e4-897d-20c352cba1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:14.586191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad635e0-323c-43e4-897d-20c352cba1c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:14.604400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad635e0-323c-43e4-897d-20c352cba1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9ad635e0-323c-43e4-897d-20c352cba1c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:14.609825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:14.610721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:14.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:14.622270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 177/340] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:14.633376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad635e0-323c-43e4-897d-20c352cba1c7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:14.663796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:14.680197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:14.766529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1eba7-6149-4a84-9c6c-cc0e6774f013 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:14.773959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad635e0-323c-43e4-897d-20c352cba1c7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:14.815121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515aad2-00e6-4a64-aab8-78281737e530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:14.816097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515aad2-00e6-4a64-aab8-78281737e530 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:14.816578 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515aad2-00e6-4a64-aab8-78281737e530 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:14.817050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515aad2-00e6-4a64-aab8-78281737e530 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:14.818067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515aad2-00e6-4a64-aab8-78281737e530 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:14.818589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515aad2-00e6-4a64-aab8-78281737e530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:14.820247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515aad2-00e6-4a64-aab8-78281737e530 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:14.830299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271c86d-818e-47b2-9e00-71ea657bb546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:14.830918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c86d-818e-47b2-9e00-71ea657bb546 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:14.831224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c86d-818e-47b2-9e00-71ea657bb546 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:14.831502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c86d-818e-47b2-9e00-71ea657bb546 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:14.832190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271c86d-818e-47b2-9e00-71ea657bb546 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:14.832487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271c86d-818e-47b2-9e00-71ea657bb546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:14.833222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 164/341] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:14.833876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271c86d-818e-47b2-9e00-71ea657bb546 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:14.834969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57a26e-c9a3-4d08-ae42-43a07d2ed10d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:14.835456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57a26e-c9a3-4d08-ae42-43a07d2ed10d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:14.835923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57a26e-c9a3-4d08-ae42-43a07d2ed10d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:14.837355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 165/342] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:14.839318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57a26e-c9a3-4d08-ae42-43a07d2ed10d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:14.843528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 178/343] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:14.847016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6064d8-db10-41c5-a8ed-bd30ddd89e8c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:14.847016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6064d8-db10-41c5-a8ed-bd30ddd89e8c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:14.847016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6064d8-db10-41c5-a8ed-bd30ddd89e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:14.868368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2d6064d8-db10-41c5-a8ed-bd30ddd89e8c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:14.869570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2d6064d8-db10-41c5-a8ed-bd30ddd89e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:14.975957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57a26e-c9a3-4d08-ae42-43a07d2ed10d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:14.979243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 179/344] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:14.979837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:14.980595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6064d8-db10-41c5-a8ed-bd30ddd89e8c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:14.993888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:15.019083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec9af5-d824-44c7-9fad-20e3ab230f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:15.020055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9af5-d824-44c7-9fad-20e3ab230f92 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.020539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9af5-d824-44c7-9fad-20e3ab230f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.021010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9af5-d824-44c7-9fad-20e3ab230f92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.097998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec9af5-d824-44c7-9fad-20e3ab230f92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:15.098553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6064d8-db10-41c5-a8ed-bd30ddd89e8c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:15.099643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec9af5-d824-44c7-9fad-20e3ab230f92 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:15.142535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf7bbc-a6bf-4e10-bad6-16de80390204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:15.144000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf7bbc-a6bf-4e10-bad6-16de80390204 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.144466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf7bbc-a6bf-4e10-bad6-16de80390204 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.144945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf7bbc-a6bf-4e10-bad6-16de80390204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.216346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 166/345] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:15.218027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb15bf-030e-4054-9abd-16d87e07897e None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.218518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb15bf-030e-4054-9abd-16d87e07897e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.219107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb15bf-030e-4054-9abd-16d87e07897e None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.220727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 167/346] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:15.225910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb15bf-030e-4054-9abd-16d87e07897e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:15.232439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:15.245201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf7bbc-a6bf-4e10-bad6-16de80390204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:15.247241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf7bbc-a6bf-4e10-bad6-16de80390204 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:15.264353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:15.349336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb15bf-030e-4054-9abd-16d87e07897e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:15.381163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 180/347] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:15.382901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a066d6-0e22-47fe-ac99-fb74197231ff None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.382901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a066d6-0e22-47fe-ac99-fb74197231ff None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.383100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a066d6-0e22-47fe-ac99-fb74197231ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.392793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d521e7-a87e-4a97-9245-8c8d9cff6f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:15.393742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d521e7-a87e-4a97-9245-8c8d9cff6f29 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.394245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d521e7-a87e-4a97-9245-8c8d9cff6f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.394696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d521e7-a87e-4a97-9245-8c8d9cff6f29 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.399836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d521e7-a87e-4a97-9245-8c8d9cff6f29 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:15.401548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d521e7-a87e-4a97-9245-8c8d9cff6f29 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:15.402404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d0a066d6-0e22-47fe-ac99-fb74197231ff None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:15.403695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d0a066d6-0e22-47fe-ac99-fb74197231ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:15.403966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 168/348] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:15 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:15.404938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d521e7-a87e-4a97-9245-8c8d9cff6f29 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:15.475951 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 181/349] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:15.476818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a066d6-0e22-47fe-ac99-fb74197231ff None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:15.487395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:15.521809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d521e7-a87e-4a97-9245-8c8d9cff6f29 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:15.550682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:15.575919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90175434-9519-4982-ab3f-5dc8d5f29b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:15.576558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90175434-9519-4982-ab3f-5dc8d5f29b35 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.576835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90175434-9519-4982-ab3f-5dc8d5f29b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.577103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90175434-9519-4982-ab3f-5dc8d5f29b35 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.583702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90175434-9519-4982-ab3f-5dc8d5f29b35 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:15.585175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90175434-9519-4982-ab3f-5dc8d5f29b35 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:15.586887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 169/350] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:15 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:15.587680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90175434-9519-4982-ab3f-5dc8d5f29b35 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:15.619402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a066d6-0e22-47fe-ac99-fb74197231ff None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:15.658330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12a316-a663-41e4-94da-76ff95a37434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:15.658990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12a316-a663-41e4-94da-76ff95a37434 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.659273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12a316-a663-41e4-94da-76ff95a37434 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.659547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12a316-a663-41e4-94da-76ff95a37434 None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.666153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12a316-a663-41e4-94da-76ff95a37434 None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:15.667661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12a316-a663-41e4-94da-76ff95a37434 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:15.669954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 182/351] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:15 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:15.670887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12a316-a663-41e4-94da-76ff95a37434 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:15.785319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12a316-a663-41e4-94da-76ff95a37434 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:15.787722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:15.814487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90175434-9519-4982-ab3f-5dc8d5f29b35 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:15.821017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a9fac-e286-4144-af24-775494527caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:15.821647 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a9fac-e286-4144-af24-775494527caf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.822179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a9fac-e286-4144-af24-775494527caf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.822450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a9fac-e286-4144-af24-775494527caf None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.823604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a9fac-e286-4144-af24-775494527caf None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:15.825086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a9fac-e286-4144-af24-775494527caf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:15.828208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 183/352] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:15 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:15.829012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a9fac-e286-4144-af24-775494527caf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:15.855070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4415f4-85f8-4059-a0c0-f038ee4ae59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:15.855702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4415f4-85f8-4059-a0c0-f038ee4ae59c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.856524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4415f4-85f8-4059-a0c0-f038ee4ae59c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.856801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4415f4-85f8-4059-a0c0-f038ee4ae59c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.868601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4415f4-85f8-4059-a0c0-f038ee4ae59c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-de4415f4-85f8-4059-a0c0-f038ee4ae59c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:15.874590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:15.875522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:15.876707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 170/353] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:15.877379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92084f97-d8fb-4edb-ad43-d26878ba635c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.877379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92084f97-d8fb-4edb-ad43-d26878ba635c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.877496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92084f97-d8fb-4edb-ad43-d26878ba635c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:15.879237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:15.883417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 171/354] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:15.884281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92084f97-d8fb-4edb-ad43-d26878ba635c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:15.953300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a9fac-e286-4144-af24-775494527caf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:15.997980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:15.998974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:15.999456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:15.999944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.001193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] RBAC: Adding query filter params (group.id=fe59e32590394738bfa3885d66e2637d, role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=772dfdcbe95541519b7335820a928256) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:16.001509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:16.003003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:16.009175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 184/355] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:15 2022] GET /identity/v3/role_assignments?group.id=fe59e32590394738bfa3885d66e2637d&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:16.009994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:16.032725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92084f97-d8fb-4edb-ad43-d26878ba635c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:16.086072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:16.101857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a02823-2b41-4e98-88a4-0d90dbb8783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:16.103147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02823-2b41-4e98-88a4-0d90dbb8783c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.103431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02823-2b41-4e98-88a4-0d90dbb8783c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.103864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02823-2b41-4e98-88a4-0d90dbb8783c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.104762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a02823-2b41-4e98-88a4-0d90dbb8783c None admin] RBAC: Adding query filter params (group.id=fe59e32590394738bfa3885d66e2637d, role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=772dfdcbe95541519b7335820a928256) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:16.105074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a02823-2b41-4e98-88a4-0d90dbb8783c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:16.107866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a02823-2b41-4e98-88a4-0d90dbb8783c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:16.114520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 172/356] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:15 2022] GET /identity/v3/role_assignments?group.id=fe59e32590394738bfa3885d66e2637d&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=3b325dcac3064617a5415a84759ef71a => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:16.118910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57961bb-ebbd-47a4-a8f8-6ac63dd956bb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.119216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57961bb-ebbd-47a4-a8f8-6ac63dd956bb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.119543 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57961bb-ebbd-47a4-a8f8-6ac63dd956bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.145303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b747ffa-ac25-45d1-97aa-bae00ae19e55 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:16.147183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c57961bb-ebbd-47a4-a8f8-6ac63dd956bb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:16.148778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c57961bb-ebbd-47a4-a8f8-6ac63dd956bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:16.188692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:16.189315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.189591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.189864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.190534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:16.190834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:16.192286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:16.202745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 185/357] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:16.207165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:16.224069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:16.234358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 173/358] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:16.235445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57961bb-ebbd-47a4-a8f8-6ac63dd956bb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:16.362336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:16.369705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb1836-961f-451a-b4aa-cb32712ae326 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:16.410612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57961bb-ebbd-47a4-a8f8-6ac63dd956bb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:16.441166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c087-df75-44c3-b968-d7c989e87b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:16.442208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c087-df75-44c3-b968-d7c989e87b57 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.442709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c087-df75-44c3-b968-d7c989e87b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.443202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c087-df75-44c3-b968-d7c989e87b57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.449824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf7bab-360c-4b59-82fd-fdca42593e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:16.450434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7bab-360c-4b59-82fd-fdca42593e00 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.450709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7bab-360c-4b59-82fd-fdca42593e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.450997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7bab-360c-4b59-82fd-fdca42593e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.462680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990c087-df75-44c3-b968-d7c989e87b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a990c087-df75-44c3-b968-d7c989e87b57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:16.469169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:16.470084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 07:55:16.470914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:16.473709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 186/359] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:16.479208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c087-df75-44c3-b968-d7c989e87b57 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:16.519989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:16.527502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf7bab-360c-4b59-82fd-fdca42593e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:16.530166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf7bab-360c-4b59-82fd-fdca42593e00 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:16.645136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 174/360] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:16.646205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf7bab-360c-4b59-82fd-fdca42593e00 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:16.670584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990c087-df75-44c3-b968-d7c989e87b57 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:16.720460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:16.721425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.721947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.722448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.723403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:16.723926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:16.725531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:16.735763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 187/361] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:16.736959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:16.758399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:16.761950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf7bab-360c-4b59-82fd-fdca42593e00 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:16.775346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:16.805100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546302e-9c7f-47d9-99ac-df10d4df6fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:16.806244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546302e-9c7f-47d9-99ac-df10d4df6fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.806526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546302e-9c7f-47d9-99ac-df10d4df6fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.806796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546302e-9c7f-47d9-99ac-df10d4df6fe8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.820505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546302e-9c7f-47d9-99ac-df10d4df6fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-1546302e-9c7f-47d9-99ac-df10d4df6fe8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:16.827491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:16.828629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:16.829638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:16.832309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 175/362] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:55:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:16.832309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bb4c0-3772-4ee1-aebe-146ec8c90288 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.832800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bb4c0-3772-4ee1-aebe-146ec8c90288 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.833078 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bb4c0-3772-4ee1-aebe-146ec8c90288 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.834783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 176/363] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:16.835383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bb4c0-3772-4ee1-aebe-146ec8c90288 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:16.859889 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835d78d-483d-4b7e-b82c-b8a77926423c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:16.907513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628810-d6ba-49d0-bb4c-40d8be8a6915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:16.908201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628810-d6ba-49d0-bb4c-40d8be8a6915 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:16.908478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628810-d6ba-49d0-bb4c-40d8be8a6915 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:16.908736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628810-d6ba-49d0-bb4c-40d8be8a6915 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:16.909885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2628810-d6ba-49d0-bb4c-40d8be8a6915 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:16.912300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2628810-d6ba-49d0-bb4c-40d8be8a6915 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:16.914261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 188/364] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:16 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:16.915389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628810-d6ba-49d0-bb4c-40d8be8a6915 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:16.980404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bb4c0-3772-4ee1-aebe-146ec8c90288 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:17.020196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac636b72-38eb-426e-974c-0d6133ada6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:17.020969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac636b72-38eb-426e-974c-0d6133ada6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:17.021244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac636b72-38eb-426e-974c-0d6133ada6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:17.021514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac636b72-38eb-426e-974c-0d6133ada6a7 None admin] PATH_INFO: `/v3/users/7848f7e8434540858529cdd31e7a68d9` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:17.022694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac636b72-38eb-426e-974c-0d6133ada6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=7848f7e8434540858529cdd31e7a68d9)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:17.024449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac636b72-38eb-426e-974c-0d6133ada6a7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:17.026348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 177/365] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:55:16 2022] GET /identity/v3/users/7848f7e8434540858529cdd31e7a68d9 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:17.027486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d2707-2e53-4205-8d8a-e8c213104164 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:17.028278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d2707-2e53-4205-8d8a-e8c213104164 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:17.028604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d2707-2e53-4205-8d8a-e8c213104164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:17.036101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:17.051019 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628810-d6ba-49d0-bb4c-40d8be8a6915 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:17.051294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-019d2707-2e53-4205-8d8a-e8c213104164 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:17.052857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-019d2707-2e53-4205-8d8a-e8c213104164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:17.073365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:17.104843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:17.105454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:17.105728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:17.106023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:17.106743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:17.107093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:17.108522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:17.117293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 189/366] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:17.118089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:17.146632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 178/367] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:17.147705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d2707-2e53-4205-8d8a-e8c213104164 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:17.247373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c6d67-c6f8-48c7-8e7c-715ce6312fdc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:17.272525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d2707-2e53-4205-8d8a-e8c213104164 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:17.296240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc79a67-b9cb-41e6-9676-869236475863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:17.296868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc79a67-b9cb-41e6-9676-869236475863 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:17.297142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc79a67-b9cb-41e6-9676-869236475863 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:17.297406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc79a67-b9cb-41e6-9676-869236475863 None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:17.299291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc79a67-b9cb-41e6-9676-869236475863 None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:17.300724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc79a67-b9cb-41e6-9676-869236475863 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:17.305245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 190/368] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:17 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:17.306832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc79a67-b9cb-41e6-9676-869236475863 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:17.328852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:17.329448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:17.329774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:17.330042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:17.330831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] RBAC: Adding query filter params (role.id=976d0bc0f2b14f26877bcc529d3645c7, user.id=7848f7e8434540858529cdd31e7a68d9, scope.system=all) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:17.331155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:17.333868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:17.345255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:17.357523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 179/369] 10.222.0.58 () {56 vars in 1308 bytes} [Tue Aug 23 07:55:17 2022] GET /identity/v3/role_assignments?user.id=7848f7e8434540858529cdd31e7a68d9&scope.system=all&role.id=976d0bc0f2b14f26877bcc529d3645c7 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:17.358274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:17.385434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:17.500482 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc79a67-b9cb-41e6-9676-869236475863 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:17.547230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11213301-0692-44fc-a1ae-ec156e7c335a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:17.590255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833df5b8-7fdf-4161-aeb7-16e962fdccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:17.590972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df5b8-7fdf-4161-aeb7-16e962fdccf2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:17.591245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df5b8-7fdf-4161-aeb7-16e962fdccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:17.591530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df5b8-7fdf-4161-aeb7-16e962fdccf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:17.617909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:17.618525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:17.618798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:17.619089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:17.619816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] RBAC: Adding query filter params (group.id=fe59e32590394738bfa3885d66e2637d, role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=5749ebcaf01649d2a355e1191671ad21) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:17.620115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:17.623430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:17.634603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 180/370] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:17 2022] GET /identity/v3/role_assignments?group.id=fe59e32590394738bfa3885d66e2637d&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=3b325dcac3064617a5415a84759ef71a => generated 260 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:17.677599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:17.699368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833df5b8-7fdf-4161-aeb7-16e962fdccf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:17.701501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833df5b8-7fdf-4161-aeb7-16e962fdccf2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:17.721467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:17.786086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:17.827911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729f91a-34fa-4ed8-b390-3149a00dc752 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:17.829011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 191/371] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:17.836110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833df5b8-7fdf-4161-aeb7-16e962fdccf2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:17.893494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e392b1-c62b-481a-ba24-e578adc7faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:17.894150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e392b1-c62b-481a-ba24-e578adc7faf7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:17.894434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e392b1-c62b-481a-ba24-e578adc7faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:17.894819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e392b1-c62b-481a-ba24-e578adc7faf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:17.912418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e392b1-c62b-481a-ba24-e578adc7faf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-74e392b1-c62b-481a-ba24-e578adc7faf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:17.917827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:17.918856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:17.919807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 181/372] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:17.930261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e392b1-c62b-481a-ba24-e578adc7faf7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:17.985531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833df5b8-7fdf-4161-aeb7-16e962fdccf2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:18.018693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:18.050093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2ee5-a9f0-48c5-bc38-11d690e2778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:18.050736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2ee5-a9f0-48c5-bc38-11d690e2778a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.051033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2ee5-a9f0-48c5-bc38-11d690e2778a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.051305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2ee5-a9f0-48c5-bc38-11d690e2778a None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.053892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e392b1-c62b-481a-ba24-e578adc7faf7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:18.055715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff2ee5-a9f0-48c5-bc38-11d690e2778a None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:18.057176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff2ee5-a9f0-48c5-bc38-11d690e2778a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:18.058845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 192/373] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:17 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:18.065196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2ee5-a9f0-48c5-bc38-11d690e2778a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:18.135693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:18.136490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.136806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.137668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.138356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:18.138660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:18.140326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:18.146890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:18.159426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 182/374] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:18.166346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:18.266525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2ee5-a9f0-48c5-bc38-11d690e2778a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:18.348242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45527ec3-bb60-43cf-9052-51b2f62eeded None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:18.355682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a3f0a-97e3-4442-829a-64fdbb1809d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:18.356859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a3f0a-97e3-4442-829a-64fdbb1809d5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.357128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a3f0a-97e3-4442-829a-64fdbb1809d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.357393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a3f0a-97e3-4442-829a-64fdbb1809d5 None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.363671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a3f0a-97e3-4442-829a-64fdbb1809d5 None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:18.365202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a3f0a-97e3-4442-829a-64fdbb1809d5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:18.367024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 193/375] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:18 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:18.391727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a3f0a-97e3-4442-829a-64fdbb1809d5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:18.397730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddc4e9-41ab-497d-a91d-a3d485870bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:18.398346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddc4e9-41ab-497d-a91d-a3d485870bea None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.398625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddc4e9-41ab-497d-a91d-a3d485870bea None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.398900 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddc4e9-41ab-497d-a91d-a3d485870bea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.472630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:18.492702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:18.500262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddc4e9-41ab-497d-a91d-a3d485870bea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:18.501744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddc4e9-41ab-497d-a91d-a3d485870bea None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:18.544724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a3f0a-97e3-4442-829a-64fdbb1809d5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:18.596497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:18.597111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.597390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.597662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.598446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] RBAC: Adding query filter params (group.id=fe59e32590394738bfa3885d66e2637d, role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=5749ebcaf01649d2a355e1191671ad21) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:18.598696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:18.600174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:18.611203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 194/376] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:18 2022] GET /identity/v3/role_assignments?group.id=fe59e32590394738bfa3885d66e2637d&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:18.641029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 183/377] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:18.646881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:18.769811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5979eb-4085-4647-8f55-13676d6687b2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:18.799215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:18.803429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef023ab-a93c-4cf3-9d7c-b2a89439fb80 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.803712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef023ab-a93c-4cf3-9d7c-b2a89439fb80 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.803983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef023ab-a93c-4cf3-9d7c-b2a89439fb80 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.805476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 184/378] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:18.820614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b01ed-42ba-4525-97ed-8d8224e04c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:18.821264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b01ed-42ba-4525-97ed-8d8224e04c4c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.821541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b01ed-42ba-4525-97ed-8d8224e04c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.821815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b01ed-42ba-4525-97ed-8d8224e04c4c None admin] PATH_INFO: `/v3/projects/ee2616c3f460471aaf732aeea2d3bd15` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.826911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b01ed-42ba-4525-97ed-8d8224e04c4c None admin] RBAC: Authorizing `identity:get_project(project_id=ee2616c3f460471aaf732aeea2d3bd15)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:18.828574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b01ed-42ba-4525-97ed-8d8224e04c4c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:18.830574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 195/379] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:18 2022] GET /identity/v3/projects/ee2616c3f460471aaf732aeea2d3bd15 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:18.836621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b01ed-42ba-4525-97ed-8d8224e04c4c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:18.836794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed23b-7ed6-46a6-a5f0-e7ff1796b58a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.837189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed23b-7ed6-46a6-a5f0-e7ff1796b58a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.837258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed23b-7ed6-46a6-a5f0-e7ff1796b58a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.839065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 185/380] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:18.840188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415e0b6-e67a-4ecf-9848-1e7cca076207 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.840989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415e0b6-e67a-4ecf-9848-1e7cca076207 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.841266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415e0b6-e67a-4ecf-9848-1e7cca076207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.864202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0415e0b6-e67a-4ecf-9848-1e7cca076207 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:18.865336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0415e0b6-e67a-4ecf-9848-1e7cca076207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:18.962895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 186/381] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:18.964999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3edf8-a71e-4703-9162-1dd38089e73a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:18.965517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3edf8-a71e-4703-9162-1dd38089e73a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:18.966045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3edf8-a71e-4703-9162-1dd38089e73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:18.974301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b01ed-42ba-4525-97ed-8d8224e04c4c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:18.989990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2eb3edf8-a71e-4703-9162-1dd38089e73a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:18.992208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2eb3edf8-a71e-4703-9162-1dd38089e73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:19.018364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcb047-47b0-495b-8e61-f9ad13bbf6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:19.023143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcb047-47b0-495b-8e61-f9ad13bbf6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:19.023143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcb047-47b0-495b-8e61-f9ad13bbf6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:19.023143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcb047-47b0-495b-8e61-f9ad13bbf6d2 None admin] PATH_INFO: `/v3/groups/48cd7492d46e4b3a94c074c6d9388498` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:19.030893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcb047-47b0-495b-8e61-f9ad13bbf6d2 None admin] RBAC: Authorizing `identity:get_group(group_id=48cd7492d46e4b3a94c074c6d9388498)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:19.032909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:19.033785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcb047-47b0-495b-8e61-f9ad13bbf6d2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:19.037167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 196/382] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:18 2022] GET /identity/v3/groups/48cd7492d46e4b3a94c074c6d9388498 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:19.045639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcb047-47b0-495b-8e61-f9ad13bbf6d2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:19.119680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 187/383] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:19.131279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:19.199536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcb047-47b0-495b-8e61-f9ad13bbf6d2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:19.241508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:19.242115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:19.242394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:19.242664 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:19.243420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] RBAC: Adding query filter params (group.id=48cd7492d46e4b3a94c074c6d9388498, role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=ee2616c3f460471aaf732aeea2d3bd15) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:19.243730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:19.245187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:19.253690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 197/384] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:19 2022] GET /identity/v3/role_assignments?group.id=48cd7492d46e4b3a94c074c6d9388498&scope.project.id=ee2616c3f460471aaf732aeea2d3bd15&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:19.343819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3edf8-a71e-4703-9162-1dd38089e73a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:19.457437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3edf8-a71e-4703-9162-1dd38089e73a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:19.493679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:19.496219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b93867-f2c1-49bd-a7eb-01ec9940adf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:19.497285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b93867-f2c1-49bd-a7eb-01ec9940adf2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:19.497777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b93867-f2c1-49bd-a7eb-01ec9940adf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:19.498249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b93867-f2c1-49bd-a7eb-01ec9940adf2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:19.517324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b93867-f2c1-49bd-a7eb-01ec9940adf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-19b93867-f2c1-49bd-a7eb-01ec9940adf2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:19.522817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:19.523794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:19.524594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:19.527461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 188/385] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:55:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:19.534178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b93867-f2c1-49bd-a7eb-01ec9940adf2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:19.554351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:19.638819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b7d4c-0305-4187-bb84-f319f07f63db None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:19.665828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:19.669528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b93867-f2c1-49bd-a7eb-01ec9940adf2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:19.711850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32712ac-d82c-4c55-b025-d1397ea1bcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:19.712403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32712ac-d82c-4c55-b025-d1397ea1bcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:19.712683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32712ac-d82c-4c55-b025-d1397ea1bcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:19.712955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32712ac-d82c-4c55-b025-d1397ea1bcc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:19.733442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32712ac-d82c-4c55-b025-d1397ea1bcc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d32712ac-d82c-4c55-b025-d1397ea1bcc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:19.739931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:19.740850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:19.741708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:19.742321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 198/386] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:19.752025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:19.752644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:19.752924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:19.753198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:19.753868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:19.754180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:19.755577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:19.758971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32712ac-d82c-4c55-b025-d1397ea1bcc2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:19.779999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 189/387] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:19.789400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:19.891064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:19.914707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b617-5296-4858-99c5-7e0f2078a421 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:19.954331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66df8-e978-4ac5-bac1-46a0d4fe6b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:19.955044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66df8-e978-4ac5-bac1-46a0d4fe6b09 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:19.955323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66df8-e978-4ac5-bac1-46a0d4fe6b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:19.955592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66df8-e978-4ac5-bac1-46a0d4fe6b09 None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:19.956730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66df8-e978-4ac5-bac1-46a0d4fe6b09 None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:19.958178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66df8-e978-4ac5-bac1-46a0d4fe6b09 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:19.959862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 190/388] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:19 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:19.968383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66df8-e978-4ac5-bac1-46a0d4fe6b09 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:19.974014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32712ac-d82c-4c55-b025-d1397ea1bcc2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:20.031629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:20.032380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:20.032567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:20.032875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:20.033638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:20.034014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:20.035434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:20.054641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 199/389] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:20.061308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:20.097139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66df8-e978-4ac5-bac1-46a0d4fe6b09 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:20.135187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:20.193546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6495-8d61-4c3e-8222-1e1fda679056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:20.194338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6495-8d61-4c3e-8222-1e1fda679056 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:20.194723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6495-8d61-4c3e-8222-1e1fda679056 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:20.195124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6495-8d61-4c3e-8222-1e1fda679056 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:20.199726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf6495-8d61-4c3e-8222-1e1fda679056 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:20.201294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf6495-8d61-4c3e-8222-1e1fda679056 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:20.203406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 191/390] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:19 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:20.211912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6495-8d61-4c3e-8222-1e1fda679056 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:20.213772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54ce8-45ec-4726-afdb-2f90af066570 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:20.253815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e39301-a7ab-4c73-9e0b-4b486b40223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:20.254674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e39301-a7ab-4c73-9e0b-4b486b40223a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:20.255265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e39301-a7ab-4c73-9e0b-4b486b40223a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:20.255404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:20.255989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e39301-a7ab-4c73-9e0b-4b486b40223a None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:20.257319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e39301-a7ab-4c73-9e0b-4b486b40223a None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:20.259053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e39301-a7ab-4c73-9e0b-4b486b40223a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:20.261460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 200/391] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:20 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:20.267256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e39301-a7ab-4c73-9e0b-4b486b40223a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:20.386322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e39301-a7ab-4c73-9e0b-4b486b40223a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:20.389485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:20.427640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6495-8d61-4c3e-8222-1e1fda679056 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:20.441186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83b8ab-8f95-4f01-a944-df8190f0fdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:20.441878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83b8ab-8f95-4f01-a944-df8190f0fdab None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:20.442154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83b8ab-8f95-4f01-a944-df8190f0fdab None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:20.442426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83b8ab-8f95-4f01-a944-df8190f0fdab None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:20.444311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83b8ab-8f95-4f01-a944-df8190f0fdab None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:20.445946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83b8ab-8f95-4f01-a944-df8190f0fdab None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:20.448515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 201/392] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:20 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:20.450216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3087-1e86-416f-949d-00783a0283d3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:20.450495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3087-1e86-416f-949d-00783a0283d3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:20.450767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c3087-1e86-416f-949d-00783a0283d3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:20.456136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 202/393] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:20.456136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c3087-1e86-416f-949d-00783a0283d3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:20.475933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff259c-0d98-4900-b6fb-d45ea93e127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:20.476951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff259c-0d98-4900-b6fb-d45ea93e127d None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:20.477429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff259c-0d98-4900-b6fb-d45ea93e127d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:20.477899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff259c-0d98-4900-b6fb-d45ea93e127d None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256/groups/fe59e32590394738bfa3885d66e2637d/roles/af615e7795ae42019a45dfe2bb205c49` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:20.482491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff259c-0d98-4900-b6fb-d45ea93e127d None admin] RBAC: Authorizing `identity:create_grant(project_id=772dfdcbe95541519b7335820a928256, group_id=fe59e32590394738bfa3885d66e2637d, role_id=af615e7795ae42019a45dfe2bb205c49)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:20.484206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff259c-0d98-4900-b6fb-d45ea93e127d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:20.570397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c3087-1e86-416f-949d-00783a0283d3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:20.584361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 192/394] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Aug 23 07:55:20 2022] PUT /identity/v3/projects/772dfdcbe95541519b7335820a928256/groups/fe59e32590394738bfa3885d66e2637d/roles/af615e7795ae42019a45dfe2bb205c49 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:20.586426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0173-c098-4774-bdc0-cebe4275bd4a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:20.587058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0173-c098-4774-bdc0-cebe4275bd4a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:20.587681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f0173-c098-4774-bdc0-cebe4275bd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:20.610540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f3fc7-01a7-40ae-aef5-b3a821598637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:20.611238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3fc7-01a7-40ae-aef5-b3a821598637 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:20.611514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3fc7-01a7-40ae-aef5-b3a821598637 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:20.611785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f3fc7-01a7-40ae-aef5-b3a821598637 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256/groups/fe59e32590394738bfa3885d66e2637d/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:20.613430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-543f0173-c098-4774-bdc0-cebe4275bd4a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:20.613786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f3fc7-01a7-40ae-aef5-b3a821598637 None admin] RBAC: Authorizing `identity:create_grant(project_id=772dfdcbe95541519b7335820a928256, group_id=fe59e32590394738bfa3885d66e2637d, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:20.615138 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-543f0173-c098-4774-bdc0-cebe4275bd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:20.615248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f3fc7-01a7-40ae-aef5-b3a821598637 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:20.693368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 203/395] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Aug 23 07:55:20 2022] PUT /identity/v3/projects/772dfdcbe95541519b7335820a928256/groups/fe59e32590394738bfa3885d66e2637d/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:55:20.716459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 193/396] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:21.003172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f3fc7-01a7-40ae-aef5-b3a821598637 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:21.043689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:21.094883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e3f90-dbab-4c1b-a3ae-c6b15fb1814c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.095414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e3f90-dbab-4c1b-a3ae-c6b15fb1814c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.095516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e3f90-dbab-4c1b-a3ae-c6b15fb1814c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.097074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 194/397] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:21.101867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30693fda-bb8c-4f78-8a34-55bf2b545db2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.102145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30693fda-bb8c-4f78-8a34-55bf2b545db2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.102467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30693fda-bb8c-4f78-8a34-55bf2b545db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.117691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:21.129751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f3fc7-01a7-40ae-aef5-b3a821598637 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:21.140010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-30693fda-bb8c-4f78-8a34-55bf2b545db2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:21.141213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-30693fda-bb8c-4f78-8a34-55bf2b545db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:21.176839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6a57a-bd55-4984-a16b-4d0fb6128db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:21.177873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6a57a-bd55-4984-a16b-4d0fb6128db9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.178356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6a57a-bd55-4984-a16b-4d0fb6128db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.178824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6a57a-bd55-4984-a16b-4d0fb6128db9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.189695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6a57a-bd55-4984-a16b-4d0fb6128db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ced6a57a-bd55-4984-a16b-4d0fb6128db9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:21.194856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:21.195964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:21.197226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:21.199193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 204/398] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:21.205548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6a57a-bd55-4984-a16b-4d0fb6128db9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:21.292372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 195/399] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:21.328013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:21.344913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6a57a-bd55-4984-a16b-4d0fb6128db9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:21.395029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:21.395645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.395927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.396199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.396867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:21.397170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:21.398541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:21.410237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 205/400] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:21.415683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30693fda-bb8c-4f78-8a34-55bf2b545db2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:21.536303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30693fda-bb8c-4f78-8a34-55bf2b545db2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:21.581084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064f3e1-d783-4896-b590-b1e2a2c0bde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:21.581632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f3e1-d783-4896-b590-b1e2a2c0bde9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.581907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f3e1-d783-4896-b590-b1e2a2c0bde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.582173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064f3e1-d783-4896-b590-b1e2a2c0bde9 None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.585686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064f3e1-d783-4896-b590-b1e2a2c0bde9 None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:21.587161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064f3e1-d783-4896-b590-b1e2a2c0bde9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:21.588754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 196/401] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:21 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:21.595350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:21.663989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064f3e1-d783-4896-b590-b1e2a2c0bde9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:21.668303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:21.726967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6240c1-555c-4918-9822-d2145591a763 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:21.797436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbbaad-0487-4c4d-b2c6-ba9352b8a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:21.798071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbbaad-0487-4c4d-b2c6-ba9352b8a8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.798348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbbaad-0487-4c4d-b2c6-ba9352b8a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.798619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbbaad-0487-4c4d-b2c6-ba9352b8a8b4 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.800507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbbaad-0487-4c4d-b2c6-ba9352b8a8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:21.801958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbbaad-0487-4c4d-b2c6-ba9352b8a8b4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:21.803618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 206/402] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:21 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:21.810438 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbbaad-0487-4c4d-b2c6-ba9352b8a8b4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:21.848388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:21.856614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064f3e1-d783-4896-b590-b1e2a2c0bde9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:21.896674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4d0a6-12cb-478b-aa0a-2efb1d1988ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:21.897761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d0a6-12cb-478b-aa0a-2efb1d1988ec None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.898223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d0a6-12cb-478b-aa0a-2efb1d1988ec None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.898770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d0a6-12cb-478b-aa0a-2efb1d1988ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.910920 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4d0a6-12cb-478b-aa0a-2efb1d1988ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-07d4d0a6-12cb-478b-aa0a-2efb1d1988ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:21.916839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:21.917805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:21.919177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:21.925857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 197/403] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:55:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:21.927328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1a5dd-08f4-423c-a3ff-84c3c966ebf2 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.927824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1a5dd-08f4-423c-a3ff-84c3c966ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.928300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1a5dd-08f4-423c-a3ff-84c3c966ebf2 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.931194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 198/404] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:21.935708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd627407-494b-4016-a62c-a85f15496b25 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.936277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd627407-494b-4016-a62c-a85f15496b25 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.936750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd627407-494b-4016-a62c-a85f15496b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.942366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbbaad-0487-4c4d-b2c6-ba9352b8a8b4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:21.957849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-dd627407-494b-4016-a62c-a85f15496b25 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:21.959407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dd627407-494b-4016-a62c-a85f15496b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:21.992155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ce547-0346-4882-8902-c6ec3de2e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:21.992890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce547-0346-4882-8902-c6ec3de2e22d None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:21.993185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce547-0346-4882-8902-c6ec3de2e22d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:21.993461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ce547-0346-4882-8902-c6ec3de2e22d None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21/groups/fe59e32590394738bfa3885d66e2637d/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:21.995632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ce547-0346-4882-8902-c6ec3de2e22d None admin] RBAC: Authorizing `identity:create_grant(project_id=5749ebcaf01649d2a355e1191671ad21, group_id=fe59e32590394738bfa3885d66e2637d, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:21.997454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ce547-0346-4882-8902-c6ec3de2e22d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:22.024030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:22.042961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 199/405] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:22.042961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd627407-494b-4016-a62c-a85f15496b25 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:22.123794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 207/406] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Aug 23 07:55:21 2022] PUT /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21/groups/fe59e32590394738bfa3885d66e2637d/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:22.151198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd627407-494b-4016-a62c-a85f15496b25 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:22.190187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:22.190824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:22.191189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:22.191462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:22.192165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:22.192559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:22.194005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:22.203914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 200/407] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:22.209949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ce547-0346-4882-8902-c6ec3de2e22d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:22.335645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:22.354307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ce547-0346-4882-8902-c6ec3de2e22d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:22.382893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:22.394582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b50fb-f5cc-41af-a7ae-caff9f2b00ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:22.395575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b50fb-f5cc-41af-a7ae-caff9f2b00ce None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:22.396066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b50fb-f5cc-41af-a7ae-caff9f2b00ce None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:22.396904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b50fb-f5cc-41af-a7ae-caff9f2b00ce None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:22.398560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b50fb-f5cc-41af-a7ae-caff9f2b00ce None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:22.401818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b50fb-f5cc-41af-a7ae-caff9f2b00ce None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:22.404761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 208/408] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:22 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:22.410634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b50fb-f5cc-41af-a7ae-caff9f2b00ce None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:22.454112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:22.546124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500000da-a723-4c31-9210-2477e5eb6da3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:22.563798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b50fb-f5cc-41af-a7ae-caff9f2b00ce None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:22.600452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd3b0d-4efe-403b-8921-c7e25941152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:22.601085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3b0d-4efe-403b-8921-c7e25941152f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:22.601362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3b0d-4efe-403b-8921-c7e25941152f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:22.601638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3b0d-4efe-403b-8921-c7e25941152f None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:22.602838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd3b0d-4efe-403b-8921-c7e25941152f None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:22.604314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd3b0d-4efe-403b-8921-c7e25941152f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:22.606803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 209/409] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:22 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:22.616566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd3b0d-4efe-403b-8921-c7e25941152f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:22.623402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dc6f6-b6e1-4a19-98c5-e913fc02a79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:22.624406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc6f6-b6e1-4a19-98c5-e913fc02a79d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:22.624901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc6f6-b6e1-4a19-98c5-e913fc02a79d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:22.625386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dc6f6-b6e1-4a19-98c5-e913fc02a79d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:22.645695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dc6f6-b6e1-4a19-98c5-e913fc02a79d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:22.655309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-695dc6f6-b6e1-4a19-98c5-e913fc02a79d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:22.659725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:22.660677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:22.661554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:22.665499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 201/410] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:22.670273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dc6f6-b6e1-4a19-98c5-e913fc02a79d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:22.789548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dc6f6-b6e1-4a19-98c5-e913fc02a79d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:22.807135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:22.832188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:22.833178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:22.833795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:22.834282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:22.835296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:22.836148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:22.839962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:22.854768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 202/411] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:22.860678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:22.900994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd3b0d-4efe-403b-8921-c7e25941152f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:22.978045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba1d79-0486-4bb3-8ad6-619e166fe7f1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:23.008139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87204258-0d6c-4369-b1bc-b753d603042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:23.008565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204258-0d6c-4369-b1bc-b753d603042a None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.008853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204258-0d6c-4369-b1bc-b753d603042a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.009122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87204258-0d6c-4369-b1bc-b753d603042a None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21/groups/fe59e32590394738bfa3885d66e2637d/roles/af615e7795ae42019a45dfe2bb205c49` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.015721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87204258-0d6c-4369-b1bc-b753d603042a None admin] RBAC: Authorizing `identity:create_grant(project_id=5749ebcaf01649d2a355e1191671ad21, group_id=fe59e32590394738bfa3885d66e2637d, role_id=af615e7795ae42019a45dfe2bb205c49)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:23.017191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87204258-0d6c-4369-b1bc-b753d603042a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:23.017741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187e13-642e-46ea-a484-4d9609bbf6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:23.018600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187e13-642e-46ea-a484-4d9609bbf6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.019134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187e13-642e-46ea-a484-4d9609bbf6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.019863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187e13-642e-46ea-a484-4d9609bbf6d4 None admin] PATH_INFO: `/v3/groups/48cd7492d46e4b3a94c074c6d9388498` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.022136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce187e13-642e-46ea-a484-4d9609bbf6d4 None admin] RBAC: Authorizing `identity:get_group(group_id=48cd7492d46e4b3a94c074c6d9388498)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:23.023996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce187e13-642e-46ea-a484-4d9609bbf6d4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:23.026334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 203/412] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:22 2022] GET /identity/v3/groups/48cd7492d46e4b3a94c074c6d9388498 => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:23.032095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187e13-642e-46ea-a484-4d9609bbf6d4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:23.081639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:23.100181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 210/413] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Aug 23 07:55:22 2022] PUT /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21/groups/fe59e32590394738bfa3885d66e2637d/roles/af615e7795ae42019a45dfe2bb205c49 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:55:23.105708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff35a5-9a64-4a85-85fb-7c2447fe68f2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.106029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff35a5-9a64-4a85-85fb-7c2447fe68f2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.106305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff35a5-9a64-4a85-85fb-7c2447fe68f2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.107989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 211/414] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:55:23.113580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe583c-5a7f-44f1-bf6c-daefab5f030b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.113860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe583c-5a7f-44f1-bf6c-daefab5f030b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.114130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe583c-5a7f-44f1-bf6c-daefab5f030b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.137603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7bfe583c-5a7f-44f1-bf6c-daefab5f030b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:23.140251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7bfe583c-5a7f-44f1-bf6c-daefab5f030b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:23.150704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187e13-642e-46ea-a484-4d9609bbf6d4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:23.213066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430af0e-66a3-4240-b60e-0512622db116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:23.214412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430af0e-66a3-4240-b60e-0512622db116 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.214870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430af0e-66a3-4240-b60e-0512622db116 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.216379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430af0e-66a3-4240-b60e-0512622db116 None admin] PATH_INFO: `/v3/projects/ee2616c3f460471aaf732aeea2d3bd15` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.218085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430af0e-66a3-4240-b60e-0512622db116 None admin] RBAC: Authorizing `identity:get_project(project_id=ee2616c3f460471aaf732aeea2d3bd15)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:23.220310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430af0e-66a3-4240-b60e-0512622db116 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:23.223000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 204/415] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:23 2022] GET /identity/v3/projects/ee2616c3f460471aaf732aeea2d3bd15 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:23.228722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aa98a-ce70-4aa1-bfa1-141fc48046ef None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.228722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aa98a-ce70-4aa1-bfa1-141fc48046ef None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.228722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aa98a-ce70-4aa1-bfa1-141fc48046ef None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.230268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 205/416] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:23.232723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aa98a-ce70-4aa1-bfa1-141fc48046ef None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:23.241734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 212/417] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:23.242854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8c94c-15f0-4688-8ed6-9800a2e81571 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.243166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8c94c-15f0-4688-8ed6-9800a2e81571 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.243440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8c94c-15f0-4688-8ed6-9800a2e81571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.267845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-35d8c94c-15f0-4688-8ed6-9800a2e81571 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:23.269447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-35d8c94c-15f0-4688-8ed6-9800a2e81571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:23.275140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:23.375496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 213/418] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:23.386845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aa98a-ce70-4aa1-bfa1-141fc48046ef None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:23.400889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:23.429051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c1cd1-28b3-41a4-88cd-7bc6b28ea400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:23.429721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1cd1-28b3-41a4-88cd-7bc6b28ea400 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.430001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1cd1-28b3-41a4-88cd-7bc6b28ea400 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.430276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1cd1-28b3-41a4-88cd-7bc6b28ea400 None admin] PATH_INFO: `/v3/projects/ee2616c3f460471aaf732aeea2d3bd15/groups/48cd7492d46e4b3a94c074c6d9388498/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.432254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c1cd1-28b3-41a4-88cd-7bc6b28ea400 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee2616c3f460471aaf732aeea2d3bd15, group_id=48cd7492d46e4b3a94c074c6d9388498, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:23.433750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c1cd1-28b3-41a4-88cd-7bc6b28ea400 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:23.516955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 206/419] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Aug 23 07:55:23 2022] PUT /identity/v3/projects/ee2616c3f460471aaf732aeea2d3bd15/groups/48cd7492d46e4b3a94c074c6d9388498/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:23.530335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8c94c-15f0-4688-8ed6-9800a2e81571 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:23.671854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8c94c-15f0-4688-8ed6-9800a2e81571 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:23.718437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f0ca3-3eeb-4db8-bc83-3337739e92c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:23.719091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f0ca3-3eeb-4db8-bc83-3337739e92c4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.719364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f0ca3-3eeb-4db8-bc83-3337739e92c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.719628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f0ca3-3eeb-4db8-bc83-3337739e92c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.726622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:23.732979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f0ca3-3eeb-4db8-bc83-3337739e92c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ed5f0ca3-3eeb-4db8-bc83-3337739e92c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:23.737714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:23.745861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:23.746695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 214/420] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:55:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:23.748378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c1cd1-28b3-41a4-88cd-7bc6b28ea400 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:23.817914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f0ca3-3eeb-4db8-bc83-3337739e92c4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:23.880563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c1cd1-28b3-41a4-88cd-7bc6b28ea400 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:23.948241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:23.948860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:23.949136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:23.949465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:23.950154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:23.950456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:23.953168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:23.976693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 207/421] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:23.979181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:24.021025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f0ca3-3eeb-4db8-bc83-3337739e92c4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:24.024574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:24.100473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f27ca1-5a43-4963-9176-24313f537b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:24.101041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f27ca1-5a43-4963-9176-24313f537b40 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:24.101328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f27ca1-5a43-4963-9176-24313f537b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:24.101605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f27ca1-5a43-4963-9176-24313f537b40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:24.119212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f27ca1-5a43-4963-9176-24313f537b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-00f27ca1-5a43-4963-9176-24313f537b40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:24.124567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:24.125428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:24.126218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 215/422] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:24.135095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f27ca1-5a43-4963-9176-24313f537b40 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:24.158883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00cb35-4abd-4b0b-b60a-4fcf6d7dad25 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:24.223725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5c77-b1a9-446d-9e52-2378db1ad37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:24.224357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5c77-b1a9-446d-9e52-2378db1ad37d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:24.224602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5c77-b1a9-446d-9e52-2378db1ad37d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:24.224873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5c77-b1a9-446d-9e52-2378db1ad37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:24.248774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:24.314513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f27ca1-5a43-4963-9176-24313f537b40 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:24.353895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5c77-b1a9-446d-9e52-2378db1ad37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:24.357094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5c77-b1a9-446d-9e52-2378db1ad37d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:24.380020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:24.380636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:24.380913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:24.381182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:24.381921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:24.382229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:24.383659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:24.404366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 216/423] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:24.408853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:24.426468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:24.532032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 208/424] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:24.534090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21e7b0-b798-4c95-9f60-d0b881d28f1d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:24.534383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21e7b0-b798-4c95-9f60-d0b881d28f1d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:24.534649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21e7b0-b798-4c95-9f60-d0b881d28f1d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:24.536218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 209/425] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:24.538027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21e7b0-b798-4c95-9f60-d0b881d28f1d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:24.570441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5529d-8b24-436e-bd4b-0da257aa4dc2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:24.597316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:24.612755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c070184-0055-4e81-b1c4-99cb326780e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:24.613299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c070184-0055-4e81-b1c4-99cb326780e8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:24.613577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c070184-0055-4e81-b1c4-99cb326780e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:24.613845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c070184-0055-4e81-b1c4-99cb326780e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:24.648015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21e7b0-b798-4c95-9f60-d0b881d28f1d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:24.687604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc65d5-c004-4b23-8a10-7af1e676442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:24.688233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc65d5-c004-4b23-8a10-7af1e676442a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:24.688508 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc65d5-c004-4b23-8a10-7af1e676442a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:24.688773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc65d5-c004-4b23-8a10-7af1e676442a None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:24.690061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc65d5-c004-4b23-8a10-7af1e676442a None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:24.692200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc65d5-c004-4b23-8a10-7af1e676442a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:24.693006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c070184-0055-4e81-b1c4-99cb326780e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:24.694572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c070184-0055-4e81-b1c4-99cb326780e8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:24.698798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 210/426] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:24 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:24.701527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d9a33-ba26-4ae1-8581-935c2cd5e2d6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:24.701809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d9a33-ba26-4ae1-8581-935c2cd5e2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:24.702082 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d9a33-ba26-4ae1-8581-935c2cd5e2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:24.709360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:24.723961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8d5d9a33-ba26-4ae1-8581-935c2cd5e2d6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:24.725823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8d5d9a33-ba26-4ae1-8581-935c2cd5e2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:24.808110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 217/427] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:24.808110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c070184-0055-4e81-b1c4-99cb326780e8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:24.817435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 211/428] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:24.818698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d9a33-ba26-4ae1-8581-935c2cd5e2d6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:24.838770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:24.993633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c070184-0055-4e81-b1c4-99cb326780e8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:25.028855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d9a33-ba26-4ae1-8581-935c2cd5e2d6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:25.051300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff112a7-c72c-4f3e-a88d-ff7ccb5061cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:25.052326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff112a7-c72c-4f3e-a88d-ff7ccb5061cd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.052747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff112a7-c72c-4f3e-a88d-ff7ccb5061cd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.053143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff112a7-c72c-4f3e-a88d-ff7ccb5061cd None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.054507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff112a7-c72c-4f3e-a88d-ff7ccb5061cd None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:25.056751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff112a7-c72c-4f3e-a88d-ff7ccb5061cd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:25.059366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 218/429] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:24 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:25.065542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff112a7-c72c-4f3e-a88d-ff7ccb5061cd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:25.091492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:25.103769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98327b6d-80d5-4501-bdd9-5654663b8158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:25.104730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327b6d-80d5-4501-bdd9-5654663b8158 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.105015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327b6d-80d5-4501-bdd9-5654663b8158 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.107414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327b6d-80d5-4501-bdd9-5654663b8158 None admin] PATH_INFO: `/v3/projects/772dfdcbe95541519b7335820a928256` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.107536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98327b6d-80d5-4501-bdd9-5654663b8158 None admin] RBAC: Authorizing `identity:get_project(project_id=772dfdcbe95541519b7335820a928256)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:25.109507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98327b6d-80d5-4501-bdd9-5654663b8158 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:25.111870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 212/430] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:24 2022] GET /identity/v3/projects/772dfdcbe95541519b7335820a928256 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:25.112919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98327b6d-80d5-4501-bdd9-5654663b8158 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:25.175999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:25.188578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff112a7-c72c-4f3e-a88d-ff7ccb5061cd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:25.223665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:25.224611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.225113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.225581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.226605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] RBAC: Adding query filter params (group.id=fe59e32590394738bfa3885d66e2637d, role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=772dfdcbe95541519b7335820a928256) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:25.227228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:25.228867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:25.236642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 219/431] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:25 2022] GET /identity/v3/role_assignments?group.id=fe59e32590394738bfa3885d66e2637d&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:25.242963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:25.243462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98327b6d-80d5-4501-bdd9-5654663b8158 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:25.291004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6aab9-a931-4816-8ef0-410390ec78c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:25.291846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6aab9-a931-4816-8ef0-410390ec78c1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.292130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6aab9-a931-4816-8ef0-410390ec78c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.292863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6aab9-a931-4816-8ef0-410390ec78c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.308196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6aab9-a931-4816-8ef0-410390ec78c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c4e6aab9-a931-4816-8ef0-410390ec78c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:25.316465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:25.317426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 07:55:25.318224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:25.323650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 213/432] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:25.325541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7d143-d104-4411-a8c6-cb4bb2ac8629 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.325825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7d143-d104-4411-a8c6-cb4bb2ac8629 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.326095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7d143-d104-4411-a8c6-cb4bb2ac8629 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.332340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 214/433] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:25.332856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7d143-d104-4411-a8c6-cb4bb2ac8629 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:25.389628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d00ab-d76a-4689-83b3-34d258abf8f5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:25.400391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:25.435829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0cb41-9203-4ce6-b2e1-85e7e3d0961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:25.436705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0cb41-9203-4ce6-b2e1-85e7e3d0961d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.437134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0cb41-9203-4ce6-b2e1-85e7e3d0961d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.437408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0cb41-9203-4ce6-b2e1-85e7e3d0961d None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.439591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0cb41-9203-4ce6-b2e1-85e7e3d0961d None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:25.441532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0cb41-9203-4ce6-b2e1-85e7e3d0961d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:25.442791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 220/434] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:25 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:25.444887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b18ef-a279-4e12-92a3-f1145287c28a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.445169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b18ef-a279-4e12-92a3-f1145287c28a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.445457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b18ef-a279-4e12-92a3-f1145287c28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.453343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7d143-d104-4411-a8c6-cb4bb2ac8629 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:25.465594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a98b18ef-a279-4e12-92a3-f1145287c28a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:25.467083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a98b18ef-a279-4e12-92a3-f1145287c28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:25.485342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:25.498408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:25.503241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.503241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.503241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.503241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:25.503241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:25.503241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:25.513696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 215/435] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:55:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:25.514602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:25.554807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 221/436] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:25.554807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b18ef-a279-4e12-92a3-f1145287c28a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:25.627835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e043be6-4cdb-462f-a9c3-39cc3fa6caa7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:25.650759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:25.663029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95231569-e029-40f4-81d2-6ee68b800fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:25.663719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231569-e029-40f4-81d2-6ee68b800fef None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.664002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231569-e029-40f4-81d2-6ee68b800fef None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.664276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95231569-e029-40f4-81d2-6ee68b800fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.665004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95231569-e029-40f4-81d2-6ee68b800fef None admin] RBAC: Adding query filter params (group.id=fe59e32590394738bfa3885d66e2637d, role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=772dfdcbe95541519b7335820a928256) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:25.665310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95231569-e029-40f4-81d2-6ee68b800fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:25.666744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95231569-e029-40f4-81d2-6ee68b800fef None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:25.673469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 216/437] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:25 2022] GET /identity/v3/role_assignments?group.id=fe59e32590394738bfa3885d66e2637d&scope.project.id=772dfdcbe95541519b7335820a928256&role.id=3b325dcac3064617a5415a84759ef71a => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:25.692570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b18ef-a279-4e12-92a3-f1145287c28a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:25.734436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75896e-5caf-4f9d-bbaa-8513c802b0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:25.735543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75896e-5caf-4f9d-bbaa-8513c802b0bd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.736222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75896e-5caf-4f9d-bbaa-8513c802b0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.736697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75896e-5caf-4f9d-bbaa-8513c802b0bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.758921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:25.778755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b254d9-d422-49cf-afd9-b4b5204644a4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.779113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b254d9-d422-49cf-afd9-b4b5204644a4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.779354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b254d9-d422-49cf-afd9-b4b5204644a4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.780959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 217/438] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:55:25.788745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfae10-47a9-4a62-bf72-d9ec268f4706 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:25.789032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfae10-47a9-4a62-bf72-d9ec268f4706 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:25.789603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfae10-47a9-4a62-bf72-d9ec268f4706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:25.814297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-78bfae10-47a9-4a62-bf72-d9ec268f4706 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:25.815536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-78bfae10-47a9-4a62-bf72-d9ec268f4706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:25.820820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75896e-5caf-4f9d-bbaa-8513c802b0bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:25.822713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75896e-5caf-4f9d-bbaa-8513c802b0bd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:25.902322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 218/439] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:25.904344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfae10-47a9-4a62-bf72-d9ec268f4706 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:25.958503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 222/440] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:25.964659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75896e-5caf-4f9d-bbaa-8513c802b0bd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.028735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:26.039748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfae10-47a9-4a62-bf72-d9ec268f4706 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.063680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:26.089337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f664b-7c44-4927-a93b-b0a42747beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.089892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f664b-7c44-4927-a93b-b0a42747beb1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.091089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f664b-7c44-4927-a93b-b0a42747beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.091089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f664b-7c44-4927-a93b-b0a42747beb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.097498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75896e-5caf-4f9d-bbaa-8513c802b0bd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.104196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f664b-7c44-4927-a93b-b0a42747beb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-3e9f664b-7c44-4927-a93b-b0a42747beb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:26.109024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:26.109950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:26.110731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 219/441] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:55:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:26.117674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f664b-7c44-4927-a93b-b0a42747beb1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.139631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5eafd3-ce9d-4274-95e8-7c437b236cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.140271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5eafd3-ce9d-4274-95e8-7c437b236cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.140546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5eafd3-ce9d-4274-95e8-7c437b236cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.140816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5eafd3-ce9d-4274-95e8-7c437b236cd6 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.142931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5eafd3-ce9d-4274-95e8-7c437b236cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:26.144415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5eafd3-ce9d-4274-95e8-7c437b236cd6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:26.146205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 223/442] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:25 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:26.151681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5eafd3-ce9d-4274-95e8-7c437b236cd6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.230471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f664b-7c44-4927-a93b-b0a42747beb1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.269819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5eafd3-ce9d-4274-95e8-7c437b236cd6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.272033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.273132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.273416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.273689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.274360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:26.274667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:26.276106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:26.287021 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 220/443] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:26.287021 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.294364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:26.312430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad173fd-6ebe-4beb-bb81-7b44f38908e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.313112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad173fd-6ebe-4beb-bb81-7b44f38908e2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.313393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad173fd-6ebe-4beb-bb81-7b44f38908e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.313666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad173fd-6ebe-4beb-bb81-7b44f38908e2 None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.314786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad173fd-6ebe-4beb-bb81-7b44f38908e2 None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:26.316278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad173fd-6ebe-4beb-bb81-7b44f38908e2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:26.317929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 224/444] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:26.319842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad173fd-6ebe-4beb-bb81-7b44f38908e2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.330926 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:26.390323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528a98d-ccd3-4240-94b9-87c56e1c782f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.417942 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad173fd-6ebe-4beb-bb81-7b44f38908e2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.424480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ea5c1-65ff-458c-824c-575a56b1740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.425103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ea5c1-65ff-458c-824c-575a56b1740a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.425385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ea5c1-65ff-458c-824c-575a56b1740a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.425656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ea5c1-65ff-458c-824c-575a56b1740a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.435631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ea5c1-65ff-458c-824c-575a56b1740a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-668ea5c1-65ff-458c-824c-575a56b1740a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:26.441358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:26.442280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:26.443129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 221/445] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:26.443802 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ea5c1-65ff-458c-824c-575a56b1740a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.452522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48d764-d64c-4bb6-8e6f-85416df8f814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.453125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48d764-d64c-4bb6-8e6f-85416df8f814 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.453400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48d764-d64c-4bb6-8e6f-85416df8f814 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.453669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48d764-d64c-4bb6-8e6f-85416df8f814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.507381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:26.520761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48d764-d64c-4bb6-8e6f-85416df8f814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:26.522198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48d764-d64c-4bb6-8e6f-85416df8f814 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:26.540605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:26.544169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ea5c1-65ff-458c-824c-575a56b1740a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.578001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.578594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.578879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.579182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.579897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] RBAC: Adding query filter params (group.id=fe59e32590394738bfa3885d66e2637d, role.id=3b325dcac3064617a5415a84759ef71a, scope.project.id=5749ebcaf01649d2a355e1191671ad21) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:26.580202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:26.581642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:26.587268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 222/446] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/role_assignments?group.id=fe59e32590394738bfa3885d66e2637d&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=3b325dcac3064617a5415a84759ef71a => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:26.588042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.633733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 225/447] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:26.635003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48d764-d64c-4bb6-8e6f-85416df8f814 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.706241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cb296-8e6c-4f6e-98a5-44c0c6746ece None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.727063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:26.744173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48d764-d64c-4bb6-8e6f-85416df8f814 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.747663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.748272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.748547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.749217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.749482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:26.749782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:26.751208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:26.760299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 223/448] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:26.766812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.792393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7af59f-640c-4e0d-bfb3-9b40b6a91d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.793048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7af59f-640c-4e0d-bfb3-9b40b6a91d82 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.793398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7af59f-640c-4e0d-bfb3-9b40b6a91d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.793667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7af59f-640c-4e0d-bfb3-9b40b6a91d82 None admin] PATH_INFO: `/v3/projects/5749ebcaf01649d2a355e1191671ad21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.794793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7af59f-640c-4e0d-bfb3-9b40b6a91d82 None admin] RBAC: Authorizing `identity:get_project(project_id=5749ebcaf01649d2a355e1191671ad21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:26.796261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7af59f-640c-4e0d-bfb3-9b40b6a91d82 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:26.798637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 226/449] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/projects/5749ebcaf01649d2a355e1191671ad21 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:26.802821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:26.803995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7af59f-640c-4e0d-bfb3-9b40b6a91d82 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.885878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f656683-d001-4a90-8c6c-6c4bd8144673 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.921475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7af59f-640c-4e0d-bfb3-9b40b6a91d82 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:26.930923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834b41-9058-44ca-bbb7-f5723d2f2b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.931555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834b41-9058-44ca-bbb7-f5723d2f2b5e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.932191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834b41-9058-44ca-bbb7-f5723d2f2b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.932474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834b41-9058-44ca-bbb7-f5723d2f2b5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.959523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb7dd9-5430-4bcb-9f7b-06ef4cfc465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:26.960074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb7dd9-5430-4bcb-9f7b-06ef4cfc465c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:26.960355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb7dd9-5430-4bcb-9f7b-06ef4cfc465c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:26.960627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb7dd9-5430-4bcb-9f7b-06ef4cfc465c None admin] PATH_INFO: `/v3/groups/fe59e32590394738bfa3885d66e2637d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:26.961752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb7dd9-5430-4bcb-9f7b-06ef4cfc465c None admin] RBAC: Authorizing `identity:get_group(group_id=fe59e32590394738bfa3885d66e2637d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:26.963283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb7dd9-5430-4bcb-9f7b-06ef4cfc465c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:26.964854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 227/450] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/groups/fe59e32590394738bfa3885d66e2637d => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:26.971065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb7dd9-5430-4bcb-9f7b-06ef4cfc465c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:26.994471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:27.005211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd834b41-9058-44ca-bbb7-f5723d2f2b5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:27.006668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd834b41-9058-44ca-bbb7-f5723d2f2b5e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:27.031438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:27.070676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb7dd9-5430-4bcb-9f7b-06ef4cfc465c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:27.105547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:27.106140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:27.106419 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:27.106688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:27.107348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 224/451] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:55:27.107446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] RBAC: Adding query filter params (group.id=fe59e32590394738bfa3885d66e2637d, role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=5749ebcaf01649d2a355e1191671ad21) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:27.107727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:27.109170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:27.113111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834b41-9058-44ca-bbb7-f5723d2f2b5e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:27.119357 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 228/452] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:26 2022] GET /identity/v3/role_assignments?group.id=fe59e32590394738bfa3885d66e2637d&scope.project.id=5749ebcaf01649d2a355e1191671ad21&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:27.216064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834b41-9058-44ca-bbb7-f5723d2f2b5e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:27.240686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:27.251818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e18e7-72ee-4062-becb-87320341d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:27.252442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e18e7-72ee-4062-becb-87320341d7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:27.252718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e18e7-72ee-4062-becb-87320341d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:27.252988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e18e7-72ee-4062-becb-87320341d7f1 None admin] PATH_INFO: `/v3/projects/ee2616c3f460471aaf732aeea2d3bd15` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:27.254112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e18e7-72ee-4062-becb-87320341d7f1 None admin] RBAC: Authorizing `identity:get_project(project_id=ee2616c3f460471aaf732aeea2d3bd15)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:27.255601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e18e7-72ee-4062-becb-87320341d7f1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:27.257579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 225/453] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:55:27 2022] GET /identity/v3/projects/ee2616c3f460471aaf732aeea2d3bd15 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:27.262763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:27.352682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:27.368091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0093314-c3de-4370-879b-3608422dc089 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:27.401997 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c3c03-8bdd-42c5-8f2f-79b344c30af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:27.403166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c3c03-8bdd-42c5-8f2f-79b344c30af1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:27.403446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c3c03-8bdd-42c5-8f2f-79b344c30af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:27.403723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c3c03-8bdd-42c5-8f2f-79b344c30af1 None admin] PATH_INFO: `/v3/groups/48cd7492d46e4b3a94c074c6d9388498` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:27.404834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c3c03-8bdd-42c5-8f2f-79b344c30af1 None admin] RBAC: Authorizing `identity:get_group(group_id=48cd7492d46e4b3a94c074c6d9388498)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:27.406271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c3c03-8bdd-42c5-8f2f-79b344c30af1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:27.407893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 229/454] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Aug 23 07:55:27 2022] GET /identity/v3/groups/48cd7492d46e4b3a94c074c6d9388498 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:27.413783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e18e7-72ee-4062-becb-87320341d7f1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:27.515350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e18e7-72ee-4062-becb-87320341d7f1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:27.548487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cfd31-60f2-4265-a455-3bec6fb41bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:27.558689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cfd31-60f2-4265-a455-3bec6fb41bba None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:27.558689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cfd31-60f2-4265-a455-3bec6fb41bba None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:27.558689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cfd31-60f2-4265-a455-3bec6fb41bba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:27.558689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cfd31-60f2-4265-a455-3bec6fb41bba None admin] RBAC: Adding query filter params (group.id=48cd7492d46e4b3a94c074c6d9388498, role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=ee2616c3f460471aaf732aeea2d3bd15) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:27.558689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cfd31-60f2-4265-a455-3bec6fb41bba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:27.558689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cfd31-60f2-4265-a455-3bec6fb41bba None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:27.559263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 226/455] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Aug 23 07:55:27 2022] GET /identity/v3/role_assignments?group.id=48cd7492d46e4b3a94c074c6d9388498&scope.project.id=ee2616c3f460471aaf732aeea2d3bd15&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:29.897016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6889895-c034-4659-9265-d8197097b87d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:29.897548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6889895-c034-4659-9265-d8197097b87d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:29.897548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6889895-c034-4659-9265-d8197097b87d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:29.899187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 230/456] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:29.905003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74f9d-9f84-4f89-b89d-87a302b24dd6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:29.905797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74f9d-9f84-4f89-b89d-87a302b24dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:29.906220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74f9d-9f84-4f89-b89d-87a302b24dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:29.935973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-77a74f9d-9f84-4f89-b89d-87a302b24dd6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:29.937464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-77a74f9d-9f84-4f89-b89d-87a302b24dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:30.007696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:30.033642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 227/457] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:30.096096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0b2ed-2e9b-41cb-9a4b-f0aa3bfd4901 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.096713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0b2ed-2e9b-41cb-9a4b-f0aa3bfd4901 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.097187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0b2ed-2e9b-41cb-9a4b-f0aa3bfd4901 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.099083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 231/458] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:30.104623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98419-fe54-43e2-ae60-484906ce7e01 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.104906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98419-fe54-43e2-ae60-484906ce7e01 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.105178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f98419-fe54-43e2-ae60-484906ce7e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.131929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-60f98419-fe54-43e2-ae60-484906ce7e01 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:30.133131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-60f98419-fe54-43e2-ae60-484906ce7e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:30.150325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594118e6-d85d-483c-9a2c-caee23378893 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.150917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594118e6-d85d-483c-9a2c-caee23378893 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.151434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594118e6-d85d-483c-9a2c-caee23378893 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.153296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 232/459] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:30.161386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1a6a-4504-4a03-8c50-8e0c15714851 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.161386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1a6a-4504-4a03-8c50-8e0c15714851 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.161386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c1a6a-4504-4a03-8c50-8e0c15714851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.195826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3f9c1a6a-4504-4a03-8c50-8e0c15714851 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:30.197619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3f9c1a6a-4504-4a03-8c50-8e0c15714851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:30.223484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 228/460] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:30.303678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 233/461] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:30.384344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f98419-fe54-43e2-ae60-484906ce7e01 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:30.500212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f98419-fe54-43e2-ae60-484906ce7e01 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:30.517170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:30.535331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:30.538730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.538730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.538730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.547033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:30.548491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:30.551727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c1a6a-4504-4a03-8c50-8e0c15714851 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:30.565097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:30.566063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:30.566913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:30.566913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:30.566913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:30.566913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 229/462] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:30.573600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:30.657787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:30.674288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c1a6a-4504-4a03-8c50-8e0c15714851 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:30.686847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96efe05-abb0-486b-ad5d-b0efa11d60b3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:30.713981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d8345-576a-4bc4-adc1-34d122734dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:30.714613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d8345-576a-4bc4-adc1-34d122734dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.714921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d8345-576a-4bc4-adc1-34d122734dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.715218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d8345-576a-4bc4-adc1-34d122734dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.725485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d8345-576a-4bc4-adc1-34d122734dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:30.726966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d8345-576a-4bc4-adc1-34d122734dd2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:30.730052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:30.731214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.731494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.731760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.732391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:30.732687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:30.734051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fc3d8345-576a-4bc4-adc1-34d122734dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:30.742144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:30.743048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:30.743939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:30.743939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:30.743939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:30.747312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 234/463] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:30.749075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddac430-22bc-4701-b965-0910be3e0b18 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.749367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddac430-22bc-4701-b965-0910be3e0b18 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.749639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddac430-22bc-4701-b965-0910be3e0b18 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.749994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 230/464] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:30.753665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:30.753798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 235/465] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:30.753798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddac430-22bc-4701-b965-0910be3e0b18 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:30.787478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:30.868204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c2ce-60de-4396-aa44-b22f43b76bdb None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:30.868431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddac430-22bc-4701-b965-0910be3e0b18 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:30.903502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f675d-be7e-48e3-a3d7-eee1095165db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:30.904123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f675d-be7e-48e3-a3d7-eee1095165db None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.904361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f675d-be7e-48e3-a3d7-eee1095165db None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.904811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f675d-be7e-48e3-a3d7-eee1095165db None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.904994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:30.905454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f675d-be7e-48e3-a3d7-eee1095165db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:30.905634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.906003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f675d-be7e-48e3-a3d7-eee1095165db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:30.906074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.906226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.907152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f675d-be7e-48e3-a3d7-eee1095165db None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:30.910042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:30.916684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:30.918702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:30.921203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 236/466] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:30.928697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13529b3-dcb4-4afa-9487-738e1db061f9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:30.928697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13529b3-dcb4-4afa-9487-738e1db061f9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:30.928697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13529b3-dcb4-4afa-9487-738e1db061f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:30.939164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:30.940061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:30.940985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:30.940985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:30.940985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:30.940985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 231/467] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:30.941212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f13529b3-dcb4-4afa-9487-738e1db061f9 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:30.941282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:30.946628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f13529b3-dcb4-4afa-9487-738e1db061f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:31.016477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 237/468] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:31.017412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13529b3-dcb4-4afa-9487-738e1db061f9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:31.039752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:31.070151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ce289-a61a-4809-ac0f-76185fcd1976 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:31.107593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3af670-3ccc-4f76-9d1d-e8cf76f1eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:31.108451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3af670-3ccc-4f76-9d1d-e8cf76f1eceb None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:31.108899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3af670-3ccc-4f76-9d1d-e8cf76f1eceb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:31.109340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3af670-3ccc-4f76-9d1d-e8cf76f1eceb None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:31.110225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3af670-3ccc-4f76-9d1d-e8cf76f1eceb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:31.111898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3af670-3ccc-4f76-9d1d-e8cf76f1eceb None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:31.123731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13529b3-dcb4-4afa-9487-738e1db061f9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:31.150359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:31.159682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c547b-7b16-49cd-b526-ded905b41dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:31.160378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c547b-7b16-49cd-b526-ded905b41dd6 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:31.160730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c547b-7b16-49cd-b526-ded905b41dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:31.161073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c547b-7b16-49cd-b526-ded905b41dd6 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:31.161785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c547b-7b16-49cd-b526-ded905b41dd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:31.163342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c547b-7b16-49cd-b526-ded905b41dd6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:31.242232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 232/469] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:55:30 2022] POST /identity/v3/users => generated 246 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:31.248334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3af670-3ccc-4f76-9d1d-e8cf76f1eceb None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:31.318764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 238/470] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:55:31 2022] POST /identity/v3/users => generated 248 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:31.320169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c547b-7b16-49cd-b526-ded905b41dd6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:31.389833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3af670-3ccc-4f76-9d1d-e8cf76f1eceb None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:31.452665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:31.453816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:31.454306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:31.454778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:31.455660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:31.456188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:31.457827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:31.461077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:31.473147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 233/471] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:31.485470 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:31.494158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c547b-7b16-49cd-b526-ded905b41dd6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:31.541704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dcb50-8d48-4595-b56f-706246e82b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:31.542347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcb50-8d48-4595-b56f-706246e82b2c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:31.542620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcb50-8d48-4595-b56f-706246e82b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:31.543453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcb50-8d48-4595-b56f-706246e82b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:31.558129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dcb50-8d48-4595-b56f-706246e82b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:31.559879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dcb50-8d48-4595-b56f-706246e82b2c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-146dcb50-8d48-4595-b56f-706246e82b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:31.579779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:31.580739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:31.581766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:31.581766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:31.581766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:31.581766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 239/472] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:31.582984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240f25e-0b54-4365-8f70-11d11c0cff5e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:31.582984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240f25e-0b54-4365-8f70-11d11c0cff5e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:31.582984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240f25e-0b54-4365-8f70-11d11c0cff5e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:31.584397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 240/473] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:31.590095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240f25e-0b54-4365-8f70-11d11c0cff5e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:31.633939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:31.637881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de429e1-80e1-4f8d-87b6-12b86c70b8b0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:31.689577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ad10c-64b3-4ceb-a991-c8338cd7002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:31.690184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad10c-64b3-4ceb-a991-c8338cd7002d None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:31.690460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad10c-64b3-4ceb-a991-c8338cd7002d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:31.690729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad10c-64b3-4ceb-a991-c8338cd7002d None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:31.691398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ad10c-64b3-4ceb-a991-c8338cd7002d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:31.692834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ad10c-64b3-4ceb-a991-c8338cd7002d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:31.710057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240f25e-0b54-4365-8f70-11d11c0cff5e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:31.747791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be739e21-be07-485c-86c9-289081b524db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:31.748745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be739e21-be07-485c-86c9-289081b524db None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:31.749224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be739e21-be07-485c-86c9-289081b524db None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:31.749692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be739e21-be07-485c-86c9-289081b524db None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:31.750697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be739e21-be07-485c-86c9-289081b524db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:31.751247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be739e21-be07-485c-86c9-289081b524db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:31.752821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be739e21-be07-485c-86c9-289081b524db None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:31.765667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 241/474] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:31.768649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbae02-8c96-4cf0-8b22-c3a63e785274 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:31.768649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbae02-8c96-4cf0-8b22-c3a63e785274 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:31.768649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbae02-8c96-4cf0-8b22-c3a63e785274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:31.787850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0fcbae02-8c96-4cf0-8b22-c3a63e785274 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:31.789047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0fcbae02-8c96-4cf0-8b22-c3a63e785274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:31.818204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 234/475] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:55:31 2022] POST /identity/v3/users => generated 248 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:31.818204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ad10c-64b3-4ceb-a991-c8338cd7002d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:31.907115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:31.934059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 242/476] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:31.981731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ad10c-64b3-4ceb-a991-c8338cd7002d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:32.023607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde9a71-dfd2-4248-8d6b-9206e06c43ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:32.024569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde9a71-dfd2-4248-8d6b-9206e06c43ad None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:32.025071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde9a71-dfd2-4248-8d6b-9206e06c43ad None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:32.025542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde9a71-dfd2-4248-8d6b-9206e06c43ad None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:32.026379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde9a71-dfd2-4248-8d6b-9206e06c43ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:32.028645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde9a71-dfd2-4248-8d6b-9206e06c43ad None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:32.122796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 235/477] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:55:31 2022] POST /identity/v3/users => generated 249 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:32.254302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbae02-8c96-4cf0-8b22-c3a63e785274 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:32.287878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:32.379539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbae02-8c96-4cf0-8b22-c3a63e785274 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:32.421128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:32.421740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:32.422016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:32.422283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:32.424442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:32.424442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:32.474020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.sql.core [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 07:55:32.474020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 07:55:32.474020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [parameters: {'id': '7c69b2b6394746ffb8e2188195294659', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 07:55:32.474020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81606) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result.read() Aug 23 07:55:32.493476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return ref.to_dict() Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:55:32.494425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application yield resource Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise self.value Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application session.commit() Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 07:55:32.495869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.session.flush() Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application compat.raise_( Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 07:55:32.496959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:55:32.497915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result.read() Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:55:32.498718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [parameters: {'id': '7c69b2b6394746ffb8e2188195294659', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:32.499527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 07:55:32.508252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:32.517105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 243/478] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Aug 23 07:55:32 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:32.517258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde9a71-dfd2-4248-8d6b-9206e06c43ad None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:32.675371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde9a71-dfd2-4248-8d6b-9206e06c43ad None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:32.687658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:32.728841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05251c58-6a29-40c0-945a-8dba696c3cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:32.729471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05251c58-6a29-40c0-945a-8dba696c3cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:32.729777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05251c58-6a29-40c0-945a-8dba696c3cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:32.730053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05251c58-6a29-40c0-945a-8dba696c3cb2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:32.745729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05251c58-6a29-40c0-945a-8dba696c3cb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-05251c58-6a29-40c0-945a-8dba696c3cb2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:32.750693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:32.751629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 07:55:32.752423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:32.760289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:32.760429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 236/479] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:55:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:32.930348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:32.944398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac80d3-3268-42d4-8f56-564a5dbe2bc4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:32.992532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8dfe1-e37a-4a8a-ad59-ea26d510da1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:32.993152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8dfe1-e37a-4a8a-ad59-ea26d510da1e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:32.993430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8dfe1-e37a-4a8a-ad59-ea26d510da1e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:32.993699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8dfe1-e37a-4a8a-ad59-ea26d510da1e None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:32.994385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8dfe1-e37a-4a8a-ad59-ea26d510da1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:32.994691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8dfe1-e37a-4a8a-ad59-ea26d510da1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:32.996137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8dfe1-e37a-4a8a-ad59-ea26d510da1e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:33.010202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 244/480] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:33.487151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e51f-e161-4e23-a50d-5cc1ef0cf601 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:33.488169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e51f-e161-4e23-a50d-5cc1ef0cf601 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:33.488664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e51f-e161-4e23-a50d-5cc1ef0cf601 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:33.490435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 237/481] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:33.496521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ffbe3-bd2c-46f5-b8d3-6536b7a7fa79 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:33.497020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ffbe3-bd2c-46f5-b8d3-6536b7a7fa79 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:33.497483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ffbe3-bd2c-46f5-b8d3-6536b7a7fa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:33.516400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0a1ffbe3-bd2c-46f5-b8d3-6536b7a7fa79 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:33.517899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0a1ffbe3-bd2c-46f5-b8d3-6536b7a7fa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:33.610440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 245/482] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:33.617938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b722fdf-6fc8-41bc-b023-6644b0e8c149 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:33.618479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b722fdf-6fc8-41bc-b023-6644b0e8c149 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:33.618970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b722fdf-6fc8-41bc-b023-6644b0e8c149 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:33.620582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 238/483] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:33.626381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7a4b-1993-4514-a354-b37c7a7aec5b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:33.626895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7a4b-1993-4514-a354-b37c7a7aec5b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:33.627389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7a4b-1993-4514-a354-b37c7a7aec5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:33.651718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bf2d7a4b-1993-4514-a354-b37c7a7aec5b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:33.653750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bf2d7a4b-1993-4514-a354-b37c7a7aec5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:33.747282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 246/484] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:34.059176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b722fdf-6fc8-41bc-b023-6644b0e8c149 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:34.061293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d7a4b-1993-4514-a354-b37c7a7aec5b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:34.079331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:34.224417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b722fdf-6fc8-41bc-b023-6644b0e8c149 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:34.236917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d7a4b-1993-4514-a354-b37c7a7aec5b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:34.287563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670409e1-0b38-40c0-a81d-9f10a6c8cc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:34.288195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670409e1-0b38-40c0-a81d-9f10a6c8cc4d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.288475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670409e1-0b38-40c0-a81d-9f10a6c8cc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.288749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670409e1-0b38-40c0-a81d-9f10a6c8cc4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.293044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2703f-634f-433e-979a-88c6b7472942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:34.293979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2703f-634f-433e-979a-88c6b7472942 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.294458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2703f-634f-433e-979a-88c6b7472942 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.294923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2703f-634f-433e-979a-88c6b7472942 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.300626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670409e1-0b38-40c0-a81d-9f10a6c8cc4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-670409e1-0b38-40c0-a81d-9f10a6c8cc4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:34.306051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:34.307163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:34.308180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:34.309868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 239/485] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:34.310238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2703f-634f-433e-979a-88c6b7472942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:34.310889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4606e4-7a57-4d33-a16e-5af81bcc2c0f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.311203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4606e4-7a57-4d33-a16e-5af81bcc2c0f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.311474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4606e4-7a57-4d33-a16e-5af81bcc2c0f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.316824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 240/486] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:34.316824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4606e4-7a57-4d33-a16e-5af81bcc2c0f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-74b2703f-634f-433e-979a-88c6b7472942 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:34.322358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:34.323276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:34.325272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 247/487] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:34.330981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354f35f-1232-4581-9702-6956f39cbdf5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.330981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354f35f-1232-4581-9702-6956f39cbdf5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.330981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354f35f-1232-4581-9702-6956f39cbdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.352893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9354f35f-1232-4581-9702-6956f39cbdf5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:34.354883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9354f35f-1232-4581-9702-6956f39cbdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:34.358770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:34.439127 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4606e4-7a57-4d33-a16e-5af81bcc2c0f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:34.442882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:34.449156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 248/488] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:34.450749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354f35f-1232-4581-9702-6956f39cbdf5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:34.478280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:34.479105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.479453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.479788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.480850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:34.481730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:34.483261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:34.493017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 241/489] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:34.499062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:34.666674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354f35f-1232-4581-9702-6956f39cbdf5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:34.667983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:34.690596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930b6ce-e551-4815-a892-91714392f2d9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:34.713703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bab8e-d312-4a86-b2d2-34c7a5bd7971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:34.714553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bab8e-d312-4a86-b2d2-34c7a5bd7971 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.715795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bab8e-d312-4a86-b2d2-34c7a5bd7971 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.716574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bab8e-d312-4a86-b2d2-34c7a5bd7971 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.717445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bab8e-d312-4a86-b2d2-34c7a5bd7971 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:34.717884 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bab8e-d312-4a86-b2d2-34c7a5bd7971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:34.720019 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0bab8e-d312-4a86-b2d2-34c7a5bd7971 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:34.730884 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 249/490] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:34.732981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2cf5d-f22a-481b-aa4a-382e2ef0b41c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.733406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2cf5d-f22a-481b-aa4a-382e2ef0b41c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.733850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2cf5d-f22a-481b-aa4a-382e2ef0b41c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.735433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 250/491] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:34.736594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd6aa8-9f82-4b27-a28c-752933d7d5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:34.737435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6aa8-9f82-4b27-a28c-752933d7d5ff None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.737889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6aa8-9f82-4b27-a28c-752933d7d5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.738338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6aa8-9f82-4b27-a28c-752933d7d5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.740210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2cf5d-f22a-481b-aa4a-382e2ef0b41c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:34.750117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd6aa8-9f82-4b27-a28c-752933d7d5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:34.751864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd6aa8-9f82-4b27-a28c-752933d7d5ff None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-dacd6aa8-9f82-4b27-a28c-752933d7d5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:34.768644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:34.769580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:34.770400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:34.770400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:34.770400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:34.772194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 242/492] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:34.773484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3315c9-c113-4d95-b200-70ae9644b4d8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.773959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3315c9-c113-4d95-b200-70ae9644b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.774371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3315c9-c113-4d95-b200-70ae9644b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.797619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7e3315c9-c113-4d95-b200-70ae9644b4d8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:34.798809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7e3315c9-c113-4d95-b200-70ae9644b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:34.815079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:34.861843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2cf5d-f22a-481b-aa4a-382e2ef0b41c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:34.876878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 243/493] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:34.880104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3315c9-c113-4d95-b200-70ae9644b4d8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:34.905415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:34.906041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:34.906322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:34.906595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:34.917721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:34.919178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:34.934685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:34.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:34.936477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:34.936477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:34.936477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:34.936477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 251/494] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:34.936699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:34.990799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:35.032176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3315c9-c113-4d95-b200-70ae9644b4d8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.043083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d00ed6-7faa-4706-8dbf-9a6d780cc6c7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.058243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:35.079372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39947958-2621-4eff-9c18-9a8dee35f752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.079854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947958-2621-4eff-9c18-9a8dee35f752 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.080130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947958-2621-4eff-9c18-9a8dee35f752 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.080409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947958-2621-4eff-9c18-9a8dee35f752 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.085724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.087093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.087731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.088202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.089385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:35.090407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.091754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39947958-2621-4eff-9c18-9a8dee35f752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:35.093397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-39947958-2621-4eff-9c18-9a8dee35f752 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:35.096505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:35.097378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:35.098165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:35.102399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 244/495] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:35.103270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39947958-2621-4eff-9c18-9a8dee35f752 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.111253 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 252/496] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:35.112330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.211737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39947958-2621-4eff-9c18-9a8dee35f752 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.234171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d1b17-a766-4382-a9a9-d679dcbb83a3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.244440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:35.250098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.250708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.255721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.255721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.255721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:35.255721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.255721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.264968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 245/497] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:35.266011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.275859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.276470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.276750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.277026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.277721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:35.278028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.279445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.288456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 253/498] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:35.294136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.332654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:35.373034 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c3815-dbab-4816-9cd3-04bd0f713ab6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.399318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada78d96-8ffd-467b-9dda-25c3db73f58b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.408262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.409245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.409529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.410123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.419087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.420672 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.432942 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57e4f4-7131-4aca-b3da-1e682390dc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.433654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57e4f4-7131-4aca-b3da-1e682390dc50 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:35.433910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:35.434810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:55:35.436104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:35.436639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57e4f4-7131-4aca-b3da-1e682390dc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.436639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57e4f4-7131-4aca-b3da-1e682390dc50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.436755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 246/499] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:35.436755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.450418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57e4f4-7131-4aca-b3da-1e682390dc50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ba57e4f4-7131-4aca-b3da-1e682390dc50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:35.455527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:35.456390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:35.457191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 254/500] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:35.458611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57e4f4-7131-4aca-b3da-1e682390dc50 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.491771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:35.537930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e397dd7-9ee1-45e1-9dd9-c937b9cb0f98 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.548906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:35.554410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57e4f4-7131-4aca-b3da-1e682390dc50 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.571891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.574262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.574544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.574817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.582878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.584386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.588610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c2ea4-3c65-4ac5-ad1e-68f5fc1ad41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.589219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c2ea4-3c65-4ac5-ad1e-68f5fc1ad41b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.589493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c2ea4-3c65-4ac5-ad1e-68f5fc1ad41b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.589785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c2ea4-3c65-4ac5-ad1e-68f5fc1ad41b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:35.595390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:35.596344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 247/501] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:35.597246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.602119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c2ea4-3c65-4ac5-ad1e-68f5fc1ad41b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.603682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c2ea4-3c65-4ac5-ad1e-68f5fc1ad41b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b65c2ea4-3c65-4ac5-ad1e-68f5fc1ad41b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:35.618541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:35.619458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:35.620329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:35.620329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:35.620329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:35.620329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 255/502] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:35.621092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d62f1d-524c-48fb-999d-bf1fc3962285 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.621207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d62f1d-524c-48fb-999d-bf1fc3962285 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.621408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d62f1d-524c-48fb-999d-bf1fc3962285 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.622637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 256/503] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:35.623382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d62f1d-524c-48fb-999d-bf1fc3962285 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.700180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9bf89-bf15-49da-bf97-137ec6d086b4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.712232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:35.724171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d62f1d-524c-48fb-999d-bf1fc3962285 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.733489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.734083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.734357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.734630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.735356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:35.735653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.737065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.757888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.758424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.758701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.758991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.759656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:35.759956 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.761326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.770125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 257/504] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:35.770904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.783112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 248/505] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:35.784029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.784354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:35.870640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a96214-66d1-4804-b2bf-e6e21edb06cf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.884590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9755c-6628-402b-9ecf-91c55e1c8f83 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:35.903865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.904463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.904740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.905010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.905734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:35.906039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.907514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.917470 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92d5b8-84e7-40e3-9bc1-042364a561d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:35.918139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92d5b8-84e7-40e3-9bc1-042364a561d1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.918416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92d5b8-84e7-40e3-9bc1-042364a561d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.918694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92d5b8-84e7-40e3-9bc1-042364a561d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.919355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92d5b8-84e7-40e3-9bc1-042364a561d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:35.919656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92d5b8-84e7-40e3-9bc1-042364a561d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:35.921029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92d5b8-84e7-40e3-9bc1-042364a561d1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:35.932034 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 249/506] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:35.933265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c7e8-9e73-45ce-a8d5-9ffe3084e3ed None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:35.934063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c7e8-9e73-45ce-a8d5-9ffe3084e3ed None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:35.934338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c7e8-9e73-45ce-a8d5-9ffe3084e3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:35.944943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 258/507] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:35.945775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:35.952047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8136c7e8-9e73-45ce-a8d5-9ffe3084e3ed None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:35.953258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8136c7e8-9e73-45ce-a8d5-9ffe3084e3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:35.976865 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.013635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.019500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 250/508] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:36.020320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136c7e8-9e73-45ce-a8d5-9ffe3084e3ed None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.046989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfd1b4-7edf-4d2b-9561-4fcb706866a6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.082258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.083028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.083273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.083602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.094076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.095572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:36.109939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:36.110827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:36.111732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:36.111732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:36.111732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:36.111732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 259/509] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:36.111939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.132392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136c7e8-9e73-45ce-a8d5-9ffe3084e3ed None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.173535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29cbd-65c6-4039-b479-ad0e4bc99f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.174142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29cbd-65c6-4039-b479-ad0e4bc99f47 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.174419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29cbd-65c6-4039-b479-ad0e4bc99f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.174688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29cbd-65c6-4039-b479-ad0e4bc99f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.204768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.232521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e6bbf-be01-42b8-a145-230dbe1686f6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.247260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa29cbd-65c6-4039-b479-ad0e4bc99f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.248741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa29cbd-65c6-4039-b479-ad0e4bc99f47 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.253290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.267859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.268501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.268785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.269059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.276687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.278194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:36.291317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:36.292258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:36.293691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 260/510] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:36.294243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.353255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 251/511] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:36.354280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29cbd-65c6-4039-b479-ad0e4bc99f47 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.394668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c115643-a546-4efc-b02b-b03af57368da None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.412698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.427578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2ac25-375f-4eab-a254-bf5ac8aacfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.428171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2ac25-375f-4eab-a254-bf5ac8aacfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.428442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2ac25-375f-4eab-a254-bf5ac8aacfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.428733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2ac25-375f-4eab-a254-bf5ac8aacfb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.453952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29cbd-65c6-4039-b479-ad0e4bc99f47 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.477187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.488748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.489837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.489837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.490008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.490541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:36.490842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.492239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.496507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2ac25-375f-4eab-a254-bf5ac8aacfb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.497937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2ac25-375f-4eab-a254-bf5ac8aacfb1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.503604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 252/512] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:36.504410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.595763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 261/513] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:36.596608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2ac25-375f-4eab-a254-bf5ac8aacfb1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.607487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8996bfb-d0fa-445e-9d95-1c4dd7932840 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.626913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.640521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.641149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.641435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.641705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.652655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.654105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:36.669119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:36.670195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:36.671242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:36.671242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:36.671242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:36.671242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 253/514] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:36.671486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.677481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.696128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2ac25-375f-4eab-a254-bf5ac8aacfb1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.729521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.730111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.730440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.730717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.731462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:36.731764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.733190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.749776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 262/515] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:36.750559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.774914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d98a48-8183-4b81-ba21-6e48de450406 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.811178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.811807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.812089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.812368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.823015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.824970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:36.838686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:36.839984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:36.840857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:36.840857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:36.840857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:36.840857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 254/516] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:36.840857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.860090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646462a1-a3ef-41a9-ab3a-793260ad38fd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.866032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.893598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.894220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.894496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.894764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.901671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.903691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:36.905450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:36.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:36.916609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 263/517] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:36.922274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:36.941538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e47f87-0817-4982-b6c9-9601e0394569 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:36.976090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:36.976706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:36.976983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:36.977252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:36.987560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:36.989001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:37.003186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:37.004111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:37.004948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:37.004948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:37.004948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:37.004948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 255/518] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:37.005171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.020583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b46957-39cd-43eb-8f0a-c2ba85df1f3b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.053080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.053662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.053933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.054194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.054821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:37.055144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.056506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.068614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 264/519] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:37.069404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.081749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:37.103892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ce74-d2f9-4f3f-9129-c5cb7d2c1ee8 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.127580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:37.138525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d47635-64c6-49be-9d3d-2d60343aa8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.139155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d47635-64c6-49be-9d3d-2d60343aa8df None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.139440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d47635-64c6-49be-9d3d-2d60343aa8df None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.139707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d47635-64c6-49be-9d3d-2d60343aa8df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.169801 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de25e7e-be58-489f-97e7-1ad6c7f3707b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.203632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.204227 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.204509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.204778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.205413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:37.205716 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.207105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.207605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d47635-64c6-49be-9d3d-2d60343aa8df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.209062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d47635-64c6-49be-9d3d-2d60343aa8df None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.218779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 265/520] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:37.219515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.287225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:37.310110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 256/521] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:37.310927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d47635-64c6-49be-9d3d-2d60343aa8df None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.318053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e2d66-8780-4303-9062-157f471e0628 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.335192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:37.351822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.355424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.355424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.355424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.355424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:37.355424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.355838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.372846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 266/522] 10.222.0.58 () {56 vars in 1139 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:37.373614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.413668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d47635-64c6-49be-9d3d-2d60343aa8df None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.446503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.447396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.447682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.447951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.448588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:37.448908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.450308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.461366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 257/523] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:37.462197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.472308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde4090-3d16-4cea-a464-7760d5a14e07 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.497628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:37.506042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0cea2-3658-4555-8ff4-4d641c969e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.506631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0cea2-3658-4555-8ff4-4d641c969e77 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.506904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0cea2-3658-4555-8ff4-4d641c969e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.507201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0cea2-3658-4555-8ff4-4d641c969e77 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.507822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0cea2-3658-4555-8ff4-4d641c969e77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.514808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0cea2-3658-4555-8ff4-4d641c969e77 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.561349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dea15-d5ef-4c85-89ed-dfc9836036e2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.594153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.594785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.595056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.595327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.601703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.603157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:37.614893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:37.616444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 258/524] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:37.617455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.692619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 267/525] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:55:37 2022] POST /identity/v3/users => generated 247 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:37.693372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0cea2-3658-4555-8ff4-4d641c969e77 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.716860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:37.724378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325414a-4d22-4fdc-95f8-d9273a28ba19 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.730536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:37.760418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.761052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.761330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.762107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.773846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.775379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:37.792199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:37.805847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:37.806977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:37.806977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:37.806977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:37.806977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 259/526] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:37.806977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.826745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0cea2-3658-4555-8ff4-4d641c969e77 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.865874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f901c-c7fc-462a-b3af-b33caf9ccda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.866484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f901c-c7fc-462a-b3af-b33caf9ccda5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.866765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f901c-c7fc-462a-b3af-b33caf9ccda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.867080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f901c-c7fc-462a-b3af-b33caf9ccda5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.912101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96ed8d-a700-4bea-9d48-cedf5d4607d6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:37.940434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f901c-c7fc-462a-b3af-b33caf9ccda5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.941895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f901c-c7fc-462a-b3af-b33caf9ccda5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.949369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:37.950186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:37.950561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:37.950931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:37.958421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:37.960231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:37.972091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:37.973614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:37.974655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:37.975788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:37.975788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:37.975788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:37.975788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:37.975788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:37.975788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:37.975788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:37.975788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:37.976576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 260/527] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:37.977300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:37.990631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:38.047771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 268/528] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:38.048761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f901c-c7fc-462a-b3af-b33caf9ccda5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.091180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012e10-28a0-49f6-8b0c-c0c923a1ff90 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.127931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.128767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.129617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.130071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.130911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:38.131765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.133704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.152482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 261/529] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:38.153537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.159924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f901c-c7fc-462a-b3af-b33caf9ccda5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.190620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:38.196024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.196632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.196910 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.201048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.201048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:38.201048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.201048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.212856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 269/530] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:38.213695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.233156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:38.266713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f08cc-0428-432e-b5c9-298034e8356e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.302632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.303260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.303540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.303811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.304495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:38.304798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.306233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.323193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 262/531] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:38.323193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.337836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab54a3-1252-468d-8009-a58df45698f1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.374691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.375359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.375730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.376007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.386565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.388044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:38.402393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:38.403585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:38.404641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:38.404641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:38.404641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:38.407656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 270/532] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:38.408621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.432603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf1e32-08e8-4fa9-aa60-541dc4cb303c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.453927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:38.462254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:38.470435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.471120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.471400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.471676 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.472393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=03cc4435b9b0444b9f278e15398443c7) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:38.472700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.474213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.479891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 263/533] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/role_assignments?user.id=03cc4435b9b0444b9f278e15398443c7&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:38.481315 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.519795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8a0f7-619c-43df-bbb3-2b613d5a0cb1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.564503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.565403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.565833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.566252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.575098 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.576954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:38.591254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:38.593102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:38.594134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:38.594134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:38.594134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:38.594134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:38.594134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:38.594134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:38.594134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:38.594134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:38.595509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 271/534] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:38.596501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.646239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef316-4e26-4f9a-88c9-45ac19fe644e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.691228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.691845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.692126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.692656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.693854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=5e9526306889402c83f2d33b340c2dac) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:38.694179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.696201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.703497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 264/535] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/role_assignments?user.id=5e9526306889402c83f2d33b340c2dac&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:38.704385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.728213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552cecf-a656-4b70-b96a-052992a270c5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.733888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:38.804389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:38.833448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.834116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.834399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.834667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.835371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:38.835675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.837088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.860129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 272/536] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:38.869948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:38.883823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6e2c4-182f-4fd1-b998-d6f6f59108c5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:38.919103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:38.920046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:38.920549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:38.921016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:38.921894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:38.922397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:38.924071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:38.938208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 265/537] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:38.944801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:39.051510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251939a6-a47d-4b53-a857-e3f58ae93819 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:39.069347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:39.083207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd943-c62f-4f44-b5e5-088509b7edb4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:39.097767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:39.098545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:39.098926 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:39.099338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:39.100172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=6adffd03ce3a4ebfb2d7c5df3a2ca293) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:39.100602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:39.102219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:39.114166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 266/538] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/role_assignments?user.id=6adffd03ce3a4ebfb2d7c5df3a2ca293&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:39.142014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:39.142846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:39.143254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:39.143525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:39.151276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:39.152879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:39.165518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:39.166869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:39.168292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 273/539] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:39.175776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:39.310534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6b15b-b1a0-4f93-a19f-1cc1600f9362 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:39.346753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911c3ca-7847-4d5e-a586-1fbe8b46858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:39.347396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c3ca-7847-4d5e-a586-1fbe8b46858a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:39.347678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c3ca-7847-4d5e-a586-1fbe8b46858a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:39.347944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911c3ca-7847-4d5e-a586-1fbe8b46858a None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:39.348696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911c3ca-7847-4d5e-a586-1fbe8b46858a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:39.349001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911c3ca-7847-4d5e-a586-1fbe8b46858a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:39.350492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911c3ca-7847-4d5e-a586-1fbe8b46858a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:39.364222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 267/540] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:39.370572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:39.410507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:39.507902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee9bdd-c519-4c34-ac5a-6ec35970a0f7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:39.554161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540458a-af83-40c9-98f3-80284c0b60ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:39.554777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540458a-af83-40c9-98f3-80284c0b60ce None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:39.555086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540458a-af83-40c9-98f3-80284c0b60ce None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:39.555356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540458a-af83-40c9-98f3-80284c0b60ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:39.556091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540458a-af83-40c9-98f3-80284c0b60ce None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d99632164b3b4a1f95736a65da217db2) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:39.556399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540458a-af83-40c9-98f3-80284c0b60ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:39.557836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540458a-af83-40c9-98f3-80284c0b60ce None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:39.564440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 274/541] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:39 2022] GET /identity/v3/role_assignments?user.id=d99632164b3b4a1f95736a65da217db2&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:39.742886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7bd64-12d2-4495-9ff0-6fa23104eae8 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:39.743405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7bd64-12d2-4495-9ff0-6fa23104eae8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:39.743479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7bd64-12d2-4495-9ff0-6fa23104eae8 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:39.744982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 268/542] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:39.750610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b9760-0a19-42b8-ae24-0cfe9d27def1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:39.750896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b9760-0a19-42b8-ae24-0cfe9d27def1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:39.751188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b9760-0a19-42b8-ae24-0cfe9d27def1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:39.774650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-524b9760-0a19-42b8-ae24-0cfe9d27def1 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:39.775948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-524b9760-0a19-42b8-ae24-0cfe9d27def1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:39.851500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 275/543] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:40.184873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7bd64-12d2-4495-9ff0-6fa23104eae8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:40.231457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:40.293249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7bd64-12d2-4495-9ff0-6fa23104eae8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:40.330083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9476c-d3ab-43ef-a552-e58e80769b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:40.331116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9476c-d3ab-43ef-a552-e58e80769b44 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:40.331617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9476c-d3ab-43ef-a552-e58e80769b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:40.332085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9476c-d3ab-43ef-a552-e58e80769b44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:40.343589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9476c-d3ab-43ef-a552-e58e80769b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-42f9476c-d3ab-43ef-a552-e58e80769b44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:40.348605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:40.349473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:40.350268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:40.352875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 269/544] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:40.358863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b9760-0a19-42b8-ae24-0cfe9d27def1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:40.379001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:40.492443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b9760-0a19-42b8-ae24-0cfe9d27def1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:40.538073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:40.538684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:40.538974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:40.539247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:40.539909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:40.540209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:40.541577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:40.550583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 276/545] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:40.556248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9476c-d3ab-43ef-a552-e58e80769b44 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:40.658038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:40.667264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9476c-d3ab-43ef-a552-e58e80769b44 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:40.712378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:40.713009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:40.713288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:40.713556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:40.734214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:40.735724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:40.751323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:40.752629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:40.753477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:40.753477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:40.753477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:40.753477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 270/546] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:40.764408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:40.865118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:40.886720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8292f8e-2567-46c1-96ec-6b5c8a8a7d06 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:40.922026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b502d-5c21-407b-ad5e-8f6c41ae14fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:40.922634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b502d-5c21-407b-ad5e-8f6c41ae14fd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:40.923072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b502d-5c21-407b-ad5e-8f6c41ae14fd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:40.923345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b502d-5c21-407b-ad5e-8f6c41ae14fd None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:40.923987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b502d-5c21-407b-ad5e-8f6c41ae14fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:40.924288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b502d-5c21-407b-ad5e-8f6c41ae14fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:40.925658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b502d-5c21-407b-ad5e-8f6c41ae14fd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:40.937356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 277/547] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:40.942867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:41.024552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1128-5d51-4661-bb93-b8d90a738314 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.024846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1128-5d51-4661-bb93-b8d90a738314 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.025013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1128-5d51-4661-bb93-b8d90a738314 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.026595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 278/548] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:41.032318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802cb86-8ee5-4efa-9877-5e133bf1c92d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.032649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802cb86-8ee5-4efa-9877-5e133bf1c92d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.032874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802cb86-8ee5-4efa-9877-5e133bf1c92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.060013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9802cb86-8ee5-4efa-9877-5e133bf1c92d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:41.061217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9802cb86-8ee5-4efa-9877-5e133bf1c92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:41.081560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74000-d149-49a3-955c-472162a33012 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:41.130247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:41.131967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.132790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.133268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.140410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 279/549] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:41.141685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a98ae8-5649-4e0c-97fb-3bd84bd63ffd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.141963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a98ae8-5649-4e0c-97fb-3bd84bd63ffd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.142232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a98ae8-5649-4e0c-97fb-3bd84bd63ffd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.143787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 280/550] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:41.144574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:41.146497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:41.149219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17740-3e31-4938-b8a0-022ab01c19bc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.149497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17740-3e31-4938-b8a0-022ab01c19bc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.149766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17740-3e31-4938-b8a0-022ab01c19bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:41.161307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:41.162331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:55:41.163268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:41.167176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 271/551] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:41.170534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e7d17740-3e31-4938-b8a0-022ab01c19bc None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:41.175523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e7d17740-3e31-4938-b8a0-022ab01c19bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:41.176404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:41.220946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:41.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:41.250970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 281/552] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:41.370304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100f024-3b5b-461c-9fd8-68134f0d7328 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:41.419628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:41.420234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.420506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.420775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.421493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:41.421793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:41.423252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:41.448621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 272/553] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Aug 23 07:55:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:41.449561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d17740-3e31-4938-b8a0-022ab01c19bc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:41.454537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:41.557812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d17740-3e31-4938-b8a0-022ab01c19bc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:41.572606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:41.599327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc026305-befc-4dbe-b8e3-b63891ba82c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:41.600348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc026305-befc-4dbe-b8e3-b63891ba82c4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.600893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc026305-befc-4dbe-b8e3-b63891ba82c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.601379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc026305-befc-4dbe-b8e3-b63891ba82c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.605582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e5543-d154-4e97-90c9-b50202b31dbe None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:41.612934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc026305-befc-4dbe-b8e3-b63891ba82c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-dc026305-befc-4dbe-b8e3-b63891ba82c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:41.617930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:41.618839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:41.622410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:41.624959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 282/554] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:41.630831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc026305-befc-4dbe-b8e3-b63891ba82c4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:41.643542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bb09a-2683-406c-b1ec-2b092fc2857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:41.644153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bb09a-2683-406c-b1ec-2b092fc2857c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.644433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bb09a-2683-406c-b1ec-2b092fc2857c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.644706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bb09a-2683-406c-b1ec-2b092fc2857c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.695497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:41.718660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bb09a-2683-406c-b1ec-2b092fc2857c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:41.720486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bb09a-2683-406c-b1ec-2b092fc2857c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:41.742756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc026305-befc-4dbe-b8e3-b63891ba82c4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:41.783143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:41.783757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.784035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.784311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.784978 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:41.785374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:41.786658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:41.795209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 283/555] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:41.796011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:41.803885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:41.839717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 273/556] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:55:41.841009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08530bab-8b9b-49bc-a8cd-a7c4f9d58022 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.841294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08530bab-8b9b-49bc-a8cd-a7c4f9d58022 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.841567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08530bab-8b9b-49bc-a8cd-a7c4f9d58022 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.843504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 274/557] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:41.844232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08530bab-8b9b-49bc-a8cd-a7c4f9d58022 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:41.910924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cf61e-201b-4b9a-99ef-6bdee39cea70 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:41.934371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:41.945496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb57e98-df0e-4eed-8dc7-fc172a33677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:41.946115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57e98-df0e-4eed-8dc7-fc172a33677b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:41.947668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57e98-df0e-4eed-8dc7-fc172a33677b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:41.947668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57e98-df0e-4eed-8dc7-fc172a33677b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:41.959276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb57e98-df0e-4eed-8dc7-fc172a33677b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:41.961526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08530bab-8b9b-49bc-a8cd-a7c4f9d58022 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-acb57e98-df0e-4eed-8dc7-fc172a33677b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:41.964488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:41.965480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 284/558] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:41.966389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb57e98-df0e-4eed-8dc7-fc172a33677b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.002651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c549ae-bb08-4885-9937-f5f875db0108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.003325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c549ae-bb08-4885-9937-f5f875db0108 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.003600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c549ae-bb08-4885-9937-f5f875db0108 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.003865 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c549ae-bb08-4885-9937-f5f875db0108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.014810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c549ae-bb08-4885-9937-f5f875db0108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.016299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c549ae-bb08-4885-9937-f5f875db0108 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-61c549ae-bb08-4885-9937-f5f875db0108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:42.033280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:42.034871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:42.036266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:42.036266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:42.036266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:42.036266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 275/559] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:42.036266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bde57-f12b-4a6c-ac3d-ffd1d98480aa None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.036266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bde57-f12b-4a6c-ac3d-ffd1d98480aa None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.036266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bde57-f12b-4a6c-ac3d-ffd1d98480aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.053517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-773bde57-f12b-4a6c-ac3d-ffd1d98480aa None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:42.054725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-773bde57-f12b-4a6c-ac3d-ffd1d98480aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:42.070899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:42.076993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb57e98-df0e-4eed-8dc7-fc172a33677b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.110648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab6a34-f775-4230-90a6-13c9678e989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.111295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab6a34-f775-4230-90a6-13c9678e989f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.111571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab6a34-f775-4230-90a6-13c9678e989f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.111845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab6a34-f775-4230-90a6-13c9678e989f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.122331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 276/560] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:42.122453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab6a34-f775-4230-90a6-13c9678e989f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.123020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bde57-f12b-4a6c-ac3d-ffd1d98480aa None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.123803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab6a34-f775-4230-90a6-13c9678e989f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e1ab6a34-f775-4230-90a6-13c9678e989f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:42.154738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:42.155953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:42.156841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:42.156841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:42.156841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:42.156841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 285/561] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:42.157594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c270efc-a4c3-4611-9657-c13df38009ac None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.157882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c270efc-a4c3-4611-9657-c13df38009ac None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.158157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c270efc-a4c3-4611-9657-c13df38009ac None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.164886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 286/562] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:42.165611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c270efc-a4c3-4611-9657-c13df38009ac None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.204308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:42.228823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bde57-f12b-4a6c-ac3d-ffd1d98480aa None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.266762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.267404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.267798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.268071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.268848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:42.269154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.270527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.272428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c270efc-a4c3-4611-9657-c13df38009ac None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.283179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 277/563] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:42.283179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.299904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:42.311459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf0911-fdb3-45d3-8622-f53eff70ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.312405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf0911-fdb3-45d3-8622-f53eff70ab35 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.312405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf0911-fdb3-45d3-8622-f53eff70ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.312666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf0911-fdb3-45d3-8622-f53eff70ab35 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.313260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf0911-fdb3-45d3-8622-f53eff70ab35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:42.313564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf0911-fdb3-45d3-8622-f53eff70ab35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.314922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf0911-fdb3-45d3-8622-f53eff70ab35 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.331105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 287/564] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:42.331105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a8f98-e874-4e98-912b-ca9149c6ade1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.331105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a8f98-e874-4e98-912b-ca9149c6ade1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.331105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a8f98-e874-4e98-912b-ca9149c6ade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.345683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b01a8f98-e874-4e98-912b-ca9149c6ade1 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:42.346860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b01a8f98-e874-4e98-912b-ca9149c6ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:42.394352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417614-20a2-44d0-91c8-ee675425ef0f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.415595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 288/565] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:42.416592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a8f98-e874-4e98-912b-ca9149c6ade1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.432061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.432667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.432953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.433752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.434389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:42.434711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.436096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.441692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:42.449642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 278/566] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:42.454426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.525936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a8f98-e874-4e98-912b-ca9149c6ade1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.531499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:42.557505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b893900-11e5-47c9-89d2-8ddf26e598f7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.563661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.564298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.564574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.564845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.576397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.577837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:42.593340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:42.594503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:42.595733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:42.595733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:42.595733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:42.596205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 289/567] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:42.597868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.598559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.598904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.599267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.601487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.607180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.608615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:42.620043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:42.621669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 279/568] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:42.622860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.699270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:42.721497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832a4ea-a9f8-4744-9178-3b12ad7c3577 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.725407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbeb03-e1f3-4558-acbe-a8014e613209 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.754925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d6a2a-d9bd-47b5-885a-a7766f7d4b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.755567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6a2a-d9bd-47b5-885a-a7766f7d4b22 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.755846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6a2a-d9bd-47b5-885a-a7766f7d4b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.756115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6a2a-d9bd-47b5-885a-a7766f7d4b22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.760734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.761874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.762147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.762410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.764423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:42.768663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d6a2a-d9bd-47b5-885a-a7766f7d4b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:42.769686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.771169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f57d6a2a-d9bd-47b5-885a-a7766f7d4b22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:42.773798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:42.777082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 290/569] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:42.777911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d6a2a-d9bd-47b5-885a-a7766f7d4b22 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:42.782061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:42.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:42.783884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:42.783884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:42.783884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:42.783884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:42.783884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:42.783884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:42.783884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:42.783884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:42.787236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 280/570] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:42.787938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.880029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d6a2a-d9bd-47b5-885a-a7766f7d4b22 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.888093 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48bc49-62e9-4487-8d36-0da2726c97c7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:42.914106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.914697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.915012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.915280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.915942 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:42.916240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.918132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.920478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:42.920828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:42.921682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:42.921682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:42.921953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:42.922670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:42.922999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:42.924429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:42.929800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 291/571] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:42.931816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.941761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 281/572] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:42.942579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:42.995597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.036826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99e527-9d4b-46b4-a5e1-e1d85ccc2ef5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.041683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30efc6d-3bbf-493a-b8dc-572f78b58c7a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.070191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243be87-77d1-4691-8d33-9d3ceed4c239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.070828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243be87-77d1-4691-8d33-9d3ceed4c239 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.071133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243be87-77d1-4691-8d33-9d3ceed4c239 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.071402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243be87-77d1-4691-8d33-9d3ceed4c239 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.077056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.077658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.077935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.078210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.078871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:43.079202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.080572 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.082090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243be87-77d1-4691-8d33-9d3ceed4c239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6243be87-77d1-4691-8d33-9d3ceed4c239 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:43.087150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:43.088241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:43.089100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 292/573] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:43.090393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 282/574] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:43.090472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243be87-77d1-4691-8d33-9d3ceed4c239 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.090550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.150987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.194877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfced7-00f2-4aa8-a121-72f36e9505f9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.195454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243be87-77d1-4691-8d33-9d3ceed4c239 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.208556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.228381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.228539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.228981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.229092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.229257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.229362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.229526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.229615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] PATH_INFO: `/v3/users/glance` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.230259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:43.230561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.232022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.237091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.238557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.250133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 293/575] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:43.250533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:43.251748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:55:43.252706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:43.253261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.253332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 283/576] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:43.253332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.351145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef44d6-dce7-46fa-975f-0ffaa7b24bdf None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.353336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92141c-5340-49f1-8105-1e867e8e5e70 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.369108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.384216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.384818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.385095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.385366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.386028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:43.386385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.387203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.387826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.387933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.388106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.388378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.395567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 284/577] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:43.396360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.404490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.406307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:43.421234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:43.422352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:43.423443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:43.423443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:43.423443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:43.423443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 294/578] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:43.423443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.433427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.497258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601bd91-fdf3-47c8-8ca3-58b1185364c1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.520906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227531e-4b89-4382-8c76-bd204b5742d9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.530878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.531526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.531803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.532071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.542324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.543815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.554068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.554662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.558335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.558335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.558335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d4705311aa574da7b9f1b9d31c40cd72) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:43.558335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:43.558647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:43.559666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:43.560563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:43.560563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:43.560563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:43.560740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.560811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 285/579] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:43.560811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.563798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 295/580] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/role_assignments?user.id=d4705311aa574da7b9f1b9d31c40cd72&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:43.564685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.605599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.678113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.691617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf4643-8639-430a-b17d-1fc396e23b1a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.694669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57198f-6913-4343-9a70-cf7c72f5cae6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.729386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.730515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.730790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.731085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.731801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:43.732099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.733520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.735242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.735917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.736783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.736783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.747125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.748561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.752279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 286/581] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:43.753406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:43.769342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:43.770360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:43.771340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:43.771340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:43.771340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:43.772137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 296/582] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:43.772965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.864092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.869010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6289a-25f9-4752-acdf-d480c066ddb0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.882803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dad90-27d6-47a5-95c2-b1fa9c3de981 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:43.907496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.908428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.908849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.909126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.909771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:43.910077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.911491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.919682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:43.920518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:43.920960 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:43.921236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:43.921929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:43.922189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:43.923685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:43.924225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 287/583] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:43.924927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:43.934088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:43.945247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 297/584] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:43.946361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.029105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebca0b1-ffed-4bb2-a882-2dc2bbdc3f43 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.056041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777353a6-4b7f-4e45-aa4c-29b7c3785f58 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.062250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.062876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.063179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.063448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.076818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.078279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:44.093336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:44.094294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:44.095214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:44.095214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:44.095214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:44.095214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 288/585] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:44.096027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.098247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.098843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.099445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.099661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.100301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:44.100600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.102013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.114873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 298/586] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:44.115756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.121689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:44.162824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:44.206247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db740b8-a389-4797-8fea-3be33e5c9583 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.225319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b06c6f-9c4e-48cb-a747-a55b5b2a8e08 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.239915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.240535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.240819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.241086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.247741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.249180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.259197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.259807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.260084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:44.260199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:44.261027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:44.262471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:44.262471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:44.262471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:44.262471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:44.262471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:44.262471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:44.262471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:44.262471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:44.262927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.266490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 289/587] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:44.267356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.270010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.271474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:44.283063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:44.283981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 299/588] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:44.284898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.349010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:44.373285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6af71-d508-4e7a-afbf-6c3ae350e2fa None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.390287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80d45c-4c62-4d9a-aa70-8eacc9655884 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.390837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:44.408797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.409405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.409708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.409969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.410610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:44.410912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.412307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.424401 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 290/589] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:44.425236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.429965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.429965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.429965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.429965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.439402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.441229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:44.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:44.453767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:44.454684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 300/590] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:44.455307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.543085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ecaefb-2734-4084-9184-c19f9ddee51d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.558385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e853f4-536f-486e-b9db-0afecc520cd9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.581782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.582714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.583152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:44.583620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.584421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.585256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:44.585705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.587322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.594124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.595310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.595665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.595947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.597122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:44.597592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.599085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.604311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 291/591] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:44.604311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.627455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 301/592] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:44.628427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.645544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:44.709611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b29a-776d-4706-9bde-70fedd87f921 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.736217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab583a8-c296-48f7-8e49-27b9c74fee4a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.743524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.744123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.744395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.744663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.751701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.753132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:44.763984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:44.764881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 292/593] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:44.765791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.785977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.786662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.786958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.787298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.788062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:44.788366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.789790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.812994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 302/594] 10.222.0.58 () {56 vars in 1139 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:44.812994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:44.856391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:44.875668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:44.879425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30c6f9-b572-4089-8956-a9c35735b914 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.916760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e16557-f46f-4255-928e-4dd45509b0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.917395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bcf98-7199-4b4e-a1dc-37d9c9b3e45a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:44.917486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e16557-f46f-4255-928e-4dd45509b0d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.917714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e16557-f46f-4255-928e-4dd45509b0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.917984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e16557-f46f-4255-928e-4dd45509b0d6 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/03cc4435b9b0444b9f278e15398443c7/roles/43c2615f91944450952391b6a4dc693c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.936712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e16557-f46f-4255-928e-4dd45509b0d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=03cc4435b9b0444b9f278e15398443c7, role_id=43c2615f91944450952391b6a4dc693c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.938145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e16557-f46f-4255-928e-4dd45509b0d6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.952861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:44.953508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:44.953784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:44.954054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:44.964198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:44.966257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:44.981120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:44.982859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:44.983820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:44.983820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:44.983820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:44.983820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 303/595] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:44.984043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.047669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 293/596] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:55:44 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/03cc4435b9b0444b9f278e15398443c7/roles/43c2615f91944450952391b6a4dc693c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:45.056431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e16557-f46f-4255-928e-4dd45509b0d6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.092112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:45.100268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11adf77-0f53-41cb-a7aa-ef92429b50cc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:45.148274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:45.148900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.149179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.149447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.150153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:45.150458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:45.152512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:45.171557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 304/597] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:45.172546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.197653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e16557-f46f-4255-928e-4dd45509b0d6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:45.239219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:45.239852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.240129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.240398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.252162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:45.253671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:45.263261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:45.269598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:45.270612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:45.272091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:45.272091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:45.272091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:45.272091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 294/598] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:45.272091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.304060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250159d5-2a08-4ee9-b209-a71171307832 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:45.350129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:45.350979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.351545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.351820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.352460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:45.352762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:45.354305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:45.370311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 305/599] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:45.372040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.391806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:45.407806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11118ae9-83d3-45cd-aa48-9dabd5741274 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:45.449700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9d416-9fb8-4e92-b184-121f81b505f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:45.450462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d416-9fb8-4e92-b184-121f81b505f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.450851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d416-9fb8-4e92-b184-121f81b505f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.451272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d416-9fb8-4e92-b184-121f81b505f3 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/5e9526306889402c83f2d33b340c2dac/roles/43c2615f91944450952391b6a4dc693c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.467454 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9d416-9fb8-4e92-b184-121f81b505f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=5e9526306889402c83f2d33b340c2dac, role_id=43c2615f91944450952391b6a4dc693c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:45.469036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9d416-9fb8-4e92-b184-121f81b505f3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:45.482483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9870b6-52d2-41f9-8dd9-9c7fe6fd15b8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:45.517589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:45.518187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.518466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.518732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.519401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:45.519701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:45.521120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:45.532382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 306/600] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:45.533161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.562046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 295/601] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:55:45 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/5e9526306889402c83f2d33b340c2dac/roles/43c2615f91944450952391b6a4dc693c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:45.564357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47886c-47bd-4e91-a0e2-30b2d149310d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.567901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47886c-47bd-4e91-a0e2-30b2d149310d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.567901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47886c-47bd-4e91-a0e2-30b2d149310d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.571262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 296/602] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:45.572634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47886c-47bd-4e91-a0e2-30b2d149310d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.628065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:45.646676 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:45.663658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39112f6d-1d9b-45c5-999c-81ab92417f57 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:45.706021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6576d-0d44-4bb8-918d-853af75b62fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:45.707142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6576d-0d44-4bb8-918d-853af75b62fd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.707893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6576d-0d44-4bb8-918d-853af75b62fd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.708575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6576d-0d44-4bb8-918d-853af75b62fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.718687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47886c-47bd-4e91-a0e2-30b2d149310d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:45.719254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6576d-0d44-4bb8-918d-853af75b62fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:45.720904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6576d-0d44-4bb8-918d-853af75b62fd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-78e6576d-0d44-4bb8-918d-853af75b62fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:45.733343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:45.734292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:45.735343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:45.735343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:45.735343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:45.735343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:45.735343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:45.735343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:45.735343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:45.735343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:45.741976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 307/603] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:45.741976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc906e4-5db5-4245-ad12-eaa2211b424e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.741976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc906e4-5db5-4245-ad12-eaa2211b424e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.741976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc906e4-5db5-4245-ad12-eaa2211b424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.760966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9bc906e4-5db5-4245-ad12-eaa2211b424e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:45.762930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9bc906e4-5db5-4245-ad12-eaa2211b424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:45.767938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:45.768697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:45.769039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:45.769376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:45.779504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:45.781520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:45.797617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:45.798718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:45.799861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:45.799861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:45.799861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:45.799861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:45.799861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:45.799861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:45.799861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:45.799861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:45.800834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 297/604] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:45.801732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.840324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 308/605] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:45.841469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc906e4-5db5-4245-ad12-eaa2211b424e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:45.950935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:45.977312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2fa39-611c-46ed-b4ca-d8678274c225 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:45.991257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:46.016528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc906e4-5db5-4245-ad12-eaa2211b424e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:46.051758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:46.052513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:46.052797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:46.053070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:46.053761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:46.054069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:46.055534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:46.055643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:46.056225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:46.056503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:46.056776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:46.057460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:46.057759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:46.059824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:46.068381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 309/606] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:46.074330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:46.089431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 298/607] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:46.095255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:46.179074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea93602-3e40-4096-be9b-201c6426b13a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:46.207399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:46.215691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55799d-4248-4af9-92dd-7b4ace67509c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:46.216424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55799d-4248-4af9-92dd-7b4ace67509c None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:46.216708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55799d-4248-4af9-92dd-7b4ace67509c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:46.216982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55799d-4248-4af9-92dd-7b4ace67509c None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d99632164b3b4a1f95736a65da217db2/roles/43c2615f91944450952391b6a4dc693c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:46.220967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da36b3-5563-4fec-8bdd-dd9f4619eba6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:46.234675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55799d-4248-4af9-92dd-7b4ace67509c None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=d99632164b3b4a1f95736a65da217db2, role_id=43c2615f91944450952391b6a4dc693c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:46.236179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55799d-4248-4af9-92dd-7b4ace67509c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:46.258567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74ae4f-5036-435c-ac19-c96eac3c40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:46.259328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ae4f-5036-435c-ac19-c96eac3c40bf None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:46.259598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ae4f-5036-435c-ac19-c96eac3c40bf None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:46.259962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ae4f-5036-435c-ac19-c96eac3c40bf None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/6adffd03ce3a4ebfb2d7c5df3a2ca293/roles/43c2615f91944450952391b6a4dc693c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:46.275995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74ae4f-5036-435c-ac19-c96eac3c40bf None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=6adffd03ce3a4ebfb2d7c5df3a2ca293, role_id=43c2615f91944450952391b6a4dc693c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:46.278031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74ae4f-5036-435c-ac19-c96eac3c40bf None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:46.325775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 310/608] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:55:46 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d99632164b3b4a1f95736a65da217db2/roles/43c2615f91944450952391b6a4dc693c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:55:46.327272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55799d-4248-4af9-92dd-7b4ace67509c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:46.353651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 299/609] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:55:46 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/6adffd03ce3a4ebfb2d7c5df3a2ca293/roles/43c2615f91944450952391b6a4dc693c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:55:46.452916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55799d-4248-4af9-92dd-7b4ace67509c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:46.493701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a19d-8508-44c4-bffb-436534688bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:46.494705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a19d-8508-44c4-bffb-436534688bcf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:46.495226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a19d-8508-44c4-bffb-436534688bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:46.495740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a19d-8508-44c4-bffb-436534688bcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:46.508705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587a19d-8508-44c4-bffb-436534688bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1587a19d-8508-44c4-bffb-436534688bcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:46.514435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:46.515530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:46.516358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:46.523056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 311/610] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:46.525569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74ae4f-5036-435c-ac19-c96eac3c40bf None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:46.570741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:46.672103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74ae4f-5036-435c-ac19-c96eac3c40bf None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:46.708227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:46.708839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:46.709118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:46.709391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:46.710117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:46.710417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:46.711839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:46.720645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 300/611] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:46.726380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a19d-8508-44c4-bffb-436534688bcf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:46.763425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:46.838861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a19d-8508-44c4-bffb-436534688bcf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:46.874620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:46.875291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:46.875573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:46.875845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:46.887034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:46.888962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:46.906827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:46.907849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:46.908710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:46.908710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:46.908710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:46.908710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 312/612] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:46.916120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:47.028626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:47.048334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e194e-f588-408d-b705-81013f6ec961 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:47.098441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6fe87-cf7b-4e63-a58e-d3c2e4ec532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:47.099085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6fe87-cf7b-4e63-a58e-d3c2e4ec532a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:47.099369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6fe87-cf7b-4e63-a58e-d3c2e4ec532a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:47.099643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6fe87-cf7b-4e63-a58e-d3c2e4ec532a None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:47.100352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6fe87-cf7b-4e63-a58e-d3c2e4ec532a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:47.100660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6fe87-cf7b-4e63-a58e-d3c2e4ec532a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:47.102155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6fe87-cf7b-4e63-a58e-d3c2e4ec532a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:47.122196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 301/613] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:47.127798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:47.174047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f135-075a-45e1-9f97-a65912f0590c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:47.174331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f135-075a-45e1-9f97-a65912f0590c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:47.174603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f135-075a-45e1-9f97-a65912f0590c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:47.177053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 302/614] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:55:47.185326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccbc83-244f-43f1-b756-31f05668cdbd None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:47.185728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccbc83-244f-43f1-b756-31f05668cdbd None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:47.186476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccbc83-244f-43f1-b756-31f05668cdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:47.209346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d7ccbc83-244f-43f1-b756-31f05668cdbd None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:47.211035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d7ccbc83-244f-43f1-b756-31f05668cdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:47.260869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:47.292847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 303/615] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:47.293021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a08d7-4243-48ad-b215-028ebf02b26b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:47.338680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:47.339900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:47.340229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:47.340499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:47.348838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:47.350354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:47.362032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:47.363034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:55:47.364007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:47.367060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 313/616] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:47.372016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccbc83-244f-43f1-b756-31f05668cdbd None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:47.466266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:47.509845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccbc83-244f-43f1-b756-31f05668cdbd None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:47.558843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:47.559429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:47.559712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:47.559986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:47.560717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:47.561022 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:47.562536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:47.581276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 304/617] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Aug 23 07:55:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:47.590141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:47.657137 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:47.713133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b0db2-80a2-4dae-b623-9814bdf9a953 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:47.750917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:47.751658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:47.751911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:47.752109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:47.763991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:47.765480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:47.775321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:47.785103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:47.791969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:47.792955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:47.792955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:47.792955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:47.792955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 314/618] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:47.792955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:47.864985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12576d1b-c02b-465c-ac30-9783489a1536 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:47.914556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb599f-69da-4878-84c2-b9524776750f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:47.954437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:47.955078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:47.955366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:47.955649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:47.956566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:47.956877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:47.958264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:47.972532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:47.978164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 315/619] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:47.979602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:48.004597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf6022-b72b-497c-b3c1-be1b550ea9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:48.007667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf6022-b72b-497c-b3c1-be1b550ea9ef None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.007949 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf6022-b72b-497c-b3c1-be1b550ea9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.008218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf6022-b72b-497c-b3c1-be1b550ea9ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.025108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf6022-b72b-497c-b3c1-be1b550ea9ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-07bf6022-b72b-497c-b3c1-be1b550ea9ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:48.030955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:48.032278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:48.033118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:48.033765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 305/620] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:48.042850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be69827-d28f-4d98-9a9d-bf9cc29eb4c5 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.042850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be69827-d28f-4d98-9a9d-bf9cc29eb4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.042850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be69827-d28f-4d98-9a9d-bf9cc29eb4c5 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.042850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 306/621] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:48.044400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be69827-d28f-4d98-9a9d-bf9cc29eb4c5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:48.056569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:48.108352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cac64-7dba-4da1-858c-7502ddcd594e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:48.153834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be69827-d28f-4d98-9a9d-bf9cc29eb4c5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:48.154696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6853fe1-3dc4-4eec-a0de-a4aa8151fa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:48.155341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6853fe1-3dc4-4eec-a0de-a4aa8151fa90 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.155615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6853fe1-3dc4-4eec-a0de-a4aa8151fa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.155883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6853fe1-3dc4-4eec-a0de-a4aa8151fa90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.165902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6853fe1-3dc4-4eec-a0de-a4aa8151fa90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:48.167739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6853fe1-3dc4-4eec-a0de-a4aa8151fa90 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a6853fe1-3dc4-4eec-a0de-a4aa8151fa90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:48.179410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:48.180374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:48.181263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:48.181263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:48.181263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:48.181263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:48.181263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:48.181263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:48.181263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:48.181263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:48.188365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 316/622] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:48.189889 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5539-2b5d-49d8-a1d3-3b4c16518b1b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.190360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5539-2b5d-49d8-a1d3-3b4c16518b1b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.190832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5539-2b5d-49d8-a1d3-3b4c16518b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.194499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:48.195399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.195512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.195811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.196533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:48.196896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:48.199204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:48.210581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 307/623] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:48.211464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:48.213268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-26fb5539-2b5d-49d8-a1d3-3b4c16518b1b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:48.215802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-26fb5539-2b5d-49d8-a1d3-3b4c16518b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:48.279547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:48.303005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 317/624] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:48.304666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb5539-2b5d-49d8-a1d3-3b4c16518b1b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:48.343069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bb1c9-e175-49b3-845d-1e63d38d36ca None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:48.387727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:48.388293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.388570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.388842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.389537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:48.394607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:48.396013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:48.413773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 308/625] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:48.415111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:48.424090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:48.525519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a3b0f-48ec-43d9-bb42-fb5a89bc1f63 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:48.540198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:48.566505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb5539-2b5d-49d8-a1d3-3b4c16518b1b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:48.578124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4388aa-f9fc-477f-b992-28fb392a6638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:48.578902 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4388aa-f9fc-477f-b992-28fb392a6638 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.579198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4388aa-f9fc-477f-b992-28fb392a6638 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.579464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4388aa-f9fc-477f-b992-28fb392a6638 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d4705311aa574da7b9f1b9d31c40cd72/roles/43c2615f91944450952391b6a4dc693c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.602277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4388aa-f9fc-477f-b992-28fb392a6638 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=d4705311aa574da7b9f1b9d31c40cd72, role_id=43c2615f91944450952391b6a4dc693c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:48.603759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4388aa-f9fc-477f-b992-28fb392a6638 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:48.623282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:48.625748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.625748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.625748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.635311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:48.636892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:48.652468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:48.658448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:48.659945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:48.659945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:48.659945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:48.659945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 318/626] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:48.659945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:48.699582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 309/627] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:55:48 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d4705311aa574da7b9f1b9d31c40cd72/roles/43c2615f91944450952391b6a4dc693c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:55:48.699902 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4388aa-f9fc-477f-b992-28fb392a6638 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:48.769437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:48.773628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240d125-305a-4966-8c38-b60a0fe0057b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:48.815309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8b3e2-ac32-40e4-ae1b-3931737f710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:48.816330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8b3e2-ac32-40e4-ae1b-3931737f710d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.816820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8b3e2-ac32-40e4-ae1b-3931737f710d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.817357 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8b3e2-ac32-40e4-ae1b-3931737f710d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.831289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8b3e2-ac32-40e4-ae1b-3931737f710d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ade8b3e2-ac32-40e4-ae1b-3931737f710d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:48.836909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:48.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:48.838753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:48.841429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 319/628] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:48.842989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a09900-e292-4f52-870f-a46970726af6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.843504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a09900-e292-4f52-870f-a46970726af6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.844009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a09900-e292-4f52-870f-a46970726af6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.845795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 320/629] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:48.850320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a09900-e292-4f52-870f-a46970726af6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:48.852168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4388aa-f9fc-477f-b992-28fb392a6638 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:48.893574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6aa3-3e9f-4828-afc4-b78a1b860746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:48.894186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6aa3-3e9f-4828-afc4-b78a1b860746 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.894469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6aa3-3e9f-4828-afc4-b78a1b860746 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.894860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6aa3-3e9f-4828-afc4-b78a1b860746 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.896207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6aa3-3e9f-4828-afc4-b78a1b860746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:48.896207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6aa3-3e9f-4828-afc4-b78a1b860746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:48.897340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6aa3-3e9f-4828-afc4-b78a1b860746 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:48.909857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 310/630] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:48.915069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dd69-0d07-40e2-aab7-87278f20031b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:48.915069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dd69-0d07-40e2-aab7-87278f20031b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:48.915069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60dd69-0d07-40e2-aab7-87278f20031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:48.931680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0b60dd69-0d07-40e2-aab7-87278f20031b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:48.933164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0b60dd69-0d07-40e2-aab7-87278f20031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:48.951273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:48.966612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a09900-e292-4f52-870f-a46970726af6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.001439 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccfe86-7f5e-455c-820d-70c4aba310b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.002293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccfe86-7f5e-455c-820d-70c4aba310b8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.002576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccfe86-7f5e-455c-820d-70c4aba310b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.002843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccfe86-7f5e-455c-820d-70c4aba310b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.003533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccfe86-7f5e-455c-820d-70c4aba310b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:49.003837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccfe86-7f5e-455c-820d-70c4aba310b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:49.005260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccfe86-7f5e-455c-820d-70c4aba310b8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:49.006006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 311/631] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:49.008741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2bda-6065-4462-83c7-2aa6c5f08183 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.009308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2bda-6065-4462-83c7-2aa6c5f08183 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.009787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2bda-6065-4462-83c7-2aa6c5f08183 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.016725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 312/632] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:49.016725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f2bda-6065-4462-83c7-2aa6c5f08183 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.022321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 321/633] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:49.023941 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f4dc-d6fc-499e-bb88-3ad2f29152f7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.024457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f4dc-d6fc-499e-bb88-3ad2f29152f7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.024968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f4dc-d6fc-499e-bb88-3ad2f29152f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.045973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-19a6f4dc-d6fc-499e-bb88-3ad2f29152f7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:49.047193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-19a6f4dc-d6fc-499e-bb88-3ad2f29152f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:49.063107 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:49.130970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 322/634] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:49.130970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6f4dc-d6fc-499e-bb88-3ad2f29152f7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.205623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f2bda-6065-4462-83c7-2aa6c5f08183 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.272886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6f4dc-d6fc-499e-bb88-3ad2f29152f7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.278868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.279588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.279865 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.280132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.293409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:49.295233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:49.308466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:49.309407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:55:49.310374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:49.317151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 313/635] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:49.317151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.320457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.321085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.321363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.321648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.332559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:49.333295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:49.334079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:49.349335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:49.362617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:49.363540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:49.363540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:49.363540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:49.363540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 323/636] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:49.363540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.363540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:49.433509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ba570-fdef-4b90-99cf-aa15797b43d2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.453698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70e719-ba78-4399-a770-f5c0f6262352 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.467516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aed09-80fb-4c20-b60e-7c92d3502895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.468079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aed09-80fb-4c20-b60e-7c92d3502895 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.468348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aed09-80fb-4c20-b60e-7c92d3502895 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.468610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aed09-80fb-4c20-b60e-7c92d3502895 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.479130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0aed09-80fb-4c20-b60e-7c92d3502895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-1f0aed09-80fb-4c20-b60e-7c92d3502895 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:49.484177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:49.485318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:49.486166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 314/637] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:49.486870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aed09-80fb-4c20-b60e-7c92d3502895 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.493171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.493783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.494064 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.494336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.495090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:49.495423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:49.496850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:49.520122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 324/638] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:49.520122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.571061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:49.577364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:49.603042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aed09-80fb-4c20-b60e-7c92d3502895 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.617842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28329d-e8ee-4f8c-9193-b2900c8d5a47 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.639453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.640324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.640623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.640875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.641537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:49.641840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:49.643333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:49.653532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde51e2a-222c-4eae-89f2-8a2978a466a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.654155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51e2a-222c-4eae-89f2-8a2978a466a4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.654434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51e2a-222c-4eae-89f2-8a2978a466a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.654704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51e2a-222c-4eae-89f2-8a2978a466a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.660147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 315/639] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:49.660147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.669943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde51e2a-222c-4eae-89f2-8a2978a466a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-dde51e2a-222c-4eae-89f2-8a2978a466a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:49.674652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:49.675867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 325/640] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:49.676681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde51e2a-222c-4eae-89f2-8a2978a466a4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.771418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac9c78-2a05-43be-9669-a7e5b31346c5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.787205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde51e2a-222c-4eae-89f2-8a2978a466a4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.803584 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:49.809027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.809890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.809890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.810082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:49.810171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.810831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:49.812709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:49.812709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:49.825463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 316/641] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:49.825463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:49.830595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79576e7c-6da6-4a71-9f53-4123168eff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.831241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576e7c-6da6-4a71-9f53-4123168eff8b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.831557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576e7c-6da6-4a71-9f53-4123168eff8b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.831831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79576e7c-6da6-4a71-9f53-4123168eff8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.906948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79576e7c-6da6-4a71-9f53-4123168eff8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:49.908907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79576e7c-6da6-4a71-9f53-4123168eff8b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:49.939311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4f3a1-13a4-4a2a-b595-3b8153e6caa6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:49.975504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:49.976121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:49.976400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:49.976672 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:49.984695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:49.986244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:49.997133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:49.998119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 317/642] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:49.999101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.022096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 326/643] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:50.022906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79576e7c-6da6-4a71-9f53-4123168eff8b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.045112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.061617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.109828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d146ef-430b-4565-9296-9a57f675459f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.129691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79576e7c-6da6-4a71-9f53-4123168eff8b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.148655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.149259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.149533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.150064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.150455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:50.150756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.152700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.161453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 318/644] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:50.162079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.168804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.169434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.169710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.169979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.181277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.182728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:50.198072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:50.199212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:50.200258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:50.200258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:50.200258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:50.200258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 327/645] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:50.200258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.270620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f6248-9cbd-4d53-b0e3-c2978f686802 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.275021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.288843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.305131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a4b1e-80ff-4716-abf7-bad389184ecd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.307513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.308203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.308519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.308792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.309505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:50.309802 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.311289 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.334531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 319/646] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:50.334531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.346000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.346628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.346910 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.347211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.358761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.360363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:50.376757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:50.377943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:50.378817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:50.378817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:50.378817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:50.378817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 328/647] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:50.379082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.449678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a42e7-0c36-4bfb-8785-37bf6b504775 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.484880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.485504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.486194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.486466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.496935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.497775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9dfa7-b85c-469a-b3a4-8657224840e5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.498371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:50.513271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:50.514447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:50.515590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:50.515590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:50.515590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:50.519356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 320/648] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:50.520199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.524619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.539036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.539036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.539036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.539036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.539398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:50.539398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.539398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.549603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.557181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 329/649] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:50.557181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.632334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f1839-c9bf-44d0-ac82-d66ed38b8e6b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.670749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b44232-5116-4909-b26c-bd9d29c30739 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.671303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9286c-e2ff-4dd6-9704-8f37f079401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.671916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9286c-e2ff-4dd6-9704-8f37f079401c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.672201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9286c-e2ff-4dd6-9704-8f37f079401c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.672473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9286c-e2ff-4dd6-9704-8f37f079401c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.708116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.708725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.709006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.709279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.709943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:50.710249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.711651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.723097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 330/650] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:50.723877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.747925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9286c-e2ff-4dd6-9704-8f37f079401c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.749369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9286c-e2ff-4dd6-9704-8f37f079401c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.771228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.775610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.824631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3c520-68e6-4434-bd94-6b97e32963e6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.847945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 321/651] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:50.848989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9286c-e2ff-4dd6-9704-8f37f079401c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.860095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c519c1-1343-4d76-859c-261ba77056bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.861007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c519c1-1343-4d76-859c-261ba77056bc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.861285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c519c1-1343-4d76-859c-261ba77056bc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.861556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c519c1-1343-4d76-859c-261ba77056bc None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.862216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c519c1-1343-4d76-859c-261ba77056bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:50.862518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c519c1-1343-4d76-859c-261ba77056bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.864034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c519c1-1343-4d76-859c-261ba77056bc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:50.876044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 331/652] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:50.877216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f123-55bc-4a9b-9fca-6a830b276fbf None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.877499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f123-55bc-4a9b-9fca-6a830b276fbf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.877773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f123-55bc-4a9b-9fca-6a830b276fbf None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.878995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 332/653] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:50.879691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0f123-55bc-4a9b-9fca-6a830b276fbf None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:50.951657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9286c-e2ff-4dd6-9704-8f37f079401c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.976782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.979509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0f123-55bc-4a9b-9fca-6a830b276fbf None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:50.982011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:50.985326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:50.985963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:50.986224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:50.986493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:50.993375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:50.994817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:51.007792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:51.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:51.010161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 322/654] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:51.010853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.013385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.014008 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.014289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.014597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.021926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.023438 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:51.034472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:51.035474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 333/655] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:51.036753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.114829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd872bbf-b768-436c-ab4a-30e3a3149c1d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.133629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f6360-e897-4d1c-92ef-c8cdde0c000b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.147425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84457e52-0fe0-4eeb-9e62-bab4bd728b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.148043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84457e52-0fe0-4eeb-9e62-bab4bd728b64 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.148321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84457e52-0fe0-4eeb-9e62-bab4bd728b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.148590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84457e52-0fe0-4eeb-9e62-bab4bd728b64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.158918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84457e52-0fe0-4eeb-9e62-bab4bd728b64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.160382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84457e52-0fe0-4eeb-9e62-bab4bd728b64 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.166487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.167134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.167926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.168200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-84457e52-0fe0-4eeb-9e62-bab4bd728b64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:51.174537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:51.175499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:51.176389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:51.176389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:51.176389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:51.176569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.176651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 323/656] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:51.176721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.176788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0ea18-a74d-4628-ad6b-3c5a5bac7a86 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.177010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0ea18-a74d-4628-ad6b-3c5a5bac7a86 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.177277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0ea18-a74d-4628-ad6b-3c5a5bac7a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:51.192897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:51.194052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 334/657] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:51.195136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.197290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bcc0ea18-a74d-4628-ad6b-3c5a5bac7a86 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:51.199934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bcc0ea18-a74d-4628-ad6b-3c5a5bac7a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:51.219424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:51.228551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:51.264510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 324/658] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:51.265322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0ea18-a74d-4628-ad6b-3c5a5bac7a86 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.293895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a427-987d-403f-b612-c47fcc914294 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.329157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.329751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.330032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.330321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.331078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:51.331382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.332800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.350237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 335/659] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:51.351059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.377891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0ea18-a74d-4628-ad6b-3c5a5bac7a86 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.412991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.413691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.413869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.414140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.414822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:51.415203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.416663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.434228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 325/660] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:51.434228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.442303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:51.465239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b185e-94e6-43b4-b858-87fc8bcc545a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.465917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:51.505130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.505905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.506188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.506444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.507124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:51.507425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.508782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.524652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 336/661] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:51.524652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.543606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512dbd6-52b1-4e25-a03e-f6d94854962a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.577142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.577744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.578021 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.578287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.579041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:51.579342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.580756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.598981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 326/662] 10.222.0.58 () {56 vars in 1139 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:51.599688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.623364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5e878-320e-4fa1-ac36-ec33cdede316 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.656462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919077e-0240-4318-b313-2e2f4ed5417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.656993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919077e-0240-4318-b313-2e2f4ed5417f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.657267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919077e-0240-4318-b313-2e2f4ed5417f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.657530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919077e-0240-4318-b313-2e2f4ed5417f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.669226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:51.684372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:51.699446 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c50bf-599e-404e-ab8f-ddb3ecea4deb None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.726591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919077e-0240-4318-b313-2e2f4ed5417f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.728063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919077e-0240-4318-b313-2e2f4ed5417f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.733453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.734050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.734325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.734594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.735338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=03cc4435b9b0444b9f278e15398443c7) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:51.735643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.737066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.742646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 327/663] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/role_assignments?user.id=03cc4435b9b0444b9f278e15398443c7&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:51.743448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.847306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 337/664] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:51.848182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919077e-0240-4318-b313-2e2f4ed5417f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.876157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2f668-9a77-4beb-a996-3a3c1217510b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:51.897357 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:51.919504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:51.920236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:51.920514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:51.920786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:51.928357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:51.929809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:51.942511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:51.943502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:51.944444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:51.944444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:51.944444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:51.944444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:51.944444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:51.944444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:51.944444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:51.944444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:51.945874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 328/665] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:51.946777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:51.963081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:51.964583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919077e-0240-4318-b313-2e2f4ed5417f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.003630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc754a-7ae9-4e66-a889-f7ab0eb9c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.004261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc754a-7ae9-4e66-a889-f7ab0eb9c65e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.004540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc754a-7ae9-4e66-a889-f7ab0eb9c65e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.004816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc754a-7ae9-4e66-a889-f7ab0eb9c65e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.015465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc754a-7ae9-4e66-a889-f7ab0eb9c65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c7bc754a-7ae9-4e66-a889-f7ab0eb9c65e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:52.020084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:52.021207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:52.022054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 338/666] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:52.022753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc754a-7ae9-4e66-a889-f7ab0eb9c65e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.068381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bdb96-5cc0-4af1-9ca5-2b16c6575ba5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.105462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a86f1-9ca4-47ff-8242-0cf1e2df8077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.106068 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a86f1-9ca4-47ff-8242-0cf1e2df8077 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.106352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a86f1-9ca4-47ff-8242-0cf1e2df8077 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.106621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a86f1-9ca4-47ff-8242-0cf1e2df8077 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.123879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc754a-7ae9-4e66-a889-f7ab0eb9c65e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.125414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:52.159835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.160453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.160734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.161001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.171439 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.172817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.179860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a86f1-9ca4-47ff-8242-0cf1e2df8077 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.181094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a86f1-9ca4-47ff-8242-0cf1e2df8077 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:52.188673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:52.189614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:52.190523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:52.190523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:52.190523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:52.194225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 339/667] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:52.195015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.195338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:52.296660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 329/668] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:52.296660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a86f1-9ca4-47ff-8242-0cf1e2df8077 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.305709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb8cec-8694-40d8-9ff9-340ef05e1016 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.339972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.340576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.340855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.341130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.341846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:52.342123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.343585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.355826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 340/669] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:52.356480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.373420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:52.406881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a86f1-9ca4-47ff-8242-0cf1e2df8077 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.420340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:52.446968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.448142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.448440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.448707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.449503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:52.449799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.451179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.459311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 330/670] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:52.460092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.460631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cd734-7c7d-4166-84fc-48128c3d77b3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.498849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.499515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.499802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.500073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.500702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:52.501001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.502385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.519011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 341/671] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:52.519011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.566848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f02be7-d915-459e-b46b-aeb28bee8623 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.596604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:52.603622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.604237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.604561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.604855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.615358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.616828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.620471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61a6fa-00c6-44c9-9556-0d3dc17e639d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:52.636352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:52.637332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:52.638218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:52.638218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:52.638218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:52.642310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 331/672] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:52.643174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.655531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.656132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.656412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.656684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.657403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=5e9526306889402c83f2d33b340c2dac) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:52.657712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.659173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.665268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 342/673] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/role_assignments?user.id=5e9526306889402c83f2d33b340c2dac&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:52.665994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.675194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:52.792862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee75ba-51f6-4677-9fef-3b884fc64ce3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.802134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218ba06-4e08-4aca-8852-162afbf8fc53 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:52.833503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.834385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.834857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.835215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.843398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:52.843784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:52.844145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:52.844477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:52.847474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.849007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.851765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:52.853179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:52.855148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:52.868351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:52.869628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:52.871357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:52.871357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:52.871357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:52.872948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:52.873879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:52.874773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:52.874773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:52.874773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:52.874773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:52.874773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:52.874773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:52.874773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:52.874773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:52.877246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 332/674] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:52.877404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 343/675] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:52.877553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.880995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:52.981997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:53.008244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c1597-8d8a-4370-9e7d-24b324e2acc0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.009829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c8bf6-4ffe-45c4-98cb-c499bea9770c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.043244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.043828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.044105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.044369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.045055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:53.045377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.046835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.053066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.053675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.053950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.054687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.055367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:53.055675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.058345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.060261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 344/676] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:53.061072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:53.082505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 333/677] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:53.083452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:53.174671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26be9a-8386-4772-a4fd-81bd3217771f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.205291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff7e43-ac54-4da8-8dcb-48a1320895bb None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.221713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:53.222627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.223414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.223536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.226976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.226976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:53.226976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.226976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.243409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 345/678] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:53.244683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:53.246699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.247338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.247797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.248076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.249660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=6adffd03ce3a4ebfb2d7c5df3a2ca293) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:53.249970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.251440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.254901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:53.265382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 334/679] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:53 2022] GET /identity/v3/role_assignments?user.id=6adffd03ce3a4ebfb2d7c5df3a2ca293&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:53.265382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:53.437819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51e69a-8c9e-44a9-8830-93290b352d76 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.452184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cff8a-2b03-445c-b2a7-aef7589741b6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.462332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:53.480785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.481634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.482030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.484411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.493167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.497595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.498358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.499186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.499632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.500040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:53.507750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:53.508892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:53.510075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:53.510075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:53.510075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:53.510075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:53.510075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:53.510075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:53.510075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:53.510075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:53.512236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 346/680] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:53.512566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.514012 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.517416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:53.534977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:53.535983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:55:53.537015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:53.541094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 335/681] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:53.541887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:53.620387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:53.649626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdf8a6-ed31-4260-9534-c3d28c86ce1b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.668305 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934e58f-6c7c-4fac-a999-6464a8be561d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.695196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.696187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.696904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.697608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.699165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:53.699672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.702191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.706194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.706874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.707247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.707587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.708394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:53.708764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.710375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.718798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 347/682] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:53.726096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:53.738259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 336/683] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Aug 23 07:55:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:53.744851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:53.780099 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:53.854778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40ab1e-3593-4c94-ba44-3a00bed5d622 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.857744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf8b90-b2be-47ea-ba30-62c42809e800 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:53.877422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:53.896321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a25b83-7f97-4e25-a829-9d15fcda9199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.897037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a25b83-7f97-4e25-a829-9d15fcda9199 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.897383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a25b83-7f97-4e25-a829-9d15fcda9199 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.897798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a25b83-7f97-4e25-a829-9d15fcda9199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.899809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426de5fc-5b18-40b6-ae57-c872be12cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:53.900739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de5fc-5b18-40b6-ae57-c872be12cb6f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.901210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de5fc-5b18-40b6-ae57-c872be12cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.903090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426de5fc-5b18-40b6-ae57-c872be12cb6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.904028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426de5fc-5b18-40b6-ae57-c872be12cb6f None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d99632164b3b4a1f95736a65da217db2) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:53.904569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426de5fc-5b18-40b6-ae57-c872be12cb6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:53.906173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426de5fc-5b18-40b6-ae57-c872be12cb6f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:53.915164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 348/684] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:53 2022] GET /identity/v3/role_assignments?user.id=d99632164b3b4a1f95736a65da217db2&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:53.916870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89009f15-f0e0-45ec-9f8b-65d83cb3feea None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.917591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89009f15-f0e0-45ec-9f8b-65d83cb3feea None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.918041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89009f15-f0e0-45ec-9f8b-65d83cb3feea None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.920144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 349/685] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:53.931817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa5c07-ea8b-43ac-8f06-26ef8a7b1f1d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:53.931817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa5c07-ea8b-43ac-8f06-26ef8a7b1f1d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:53.931817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa5c07-ea8b-43ac-8f06-26ef8a7b1f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:53.969267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-57aa5c07-ea8b-43ac-8f06-26ef8a7b1f1d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:53.972971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-57aa5c07-ea8b-43ac-8f06-26ef8a7b1f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:54.019788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a25b83-7f97-4e25-a829-9d15fcda9199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:54.021475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a25b83-7f97-4e25-a829-9d15fcda9199 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:54.104404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 350/686] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 07:55:54.141244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 337/687] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 07:55:54.146712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa5c07-ea8b-43ac-8f06-26ef8a7b1f1d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:54.179461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:54.251350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa5c07-ea8b-43ac-8f06-26ef8a7b1f1d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:54.286246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:54.286875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:54.287184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:54.287457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:54.298253 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:54.299735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:54.314610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:54.315627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:54.316515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:54.316515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:54.316515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:54.316515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 351/688] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:54.322918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a25b83-7f97-4e25-a829-9d15fcda9199 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:54.345055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:54.391996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:54.487342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a25b83-7f97-4e25-a829-9d15fcda9199 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:54.536404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:54.547650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:54.549271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:54.549271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:54.549271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:54.549506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:54.549789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:54.551425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:54.556405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f77c4-da4d-4cf3-bfe3-54544b5087ff None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:54.575787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 338/689] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:54.581543 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:54.600561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079e566-b362-4987-9853-4c5937f3d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:54.601989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079e566-b362-4987-9853-4c5937f3d611 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:54.602481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079e566-b362-4987-9853-4c5937f3d611 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:54.603060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079e566-b362-4987-9853-4c5937f3d611 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:54.615911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079e566-b362-4987-9853-4c5937f3d611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a079e566-b362-4987-9853-4c5937f3d611 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:54.621991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:54.623027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:54.623880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:54.631602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 352/690] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:54.633852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079e566-b362-4987-9853-4c5937f3d611 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:54.755424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079e566-b362-4987-9853-4c5937f3d611 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:54.764480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:54.797469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:54.798282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:54.798668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:54.799094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:54.799865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:54.800484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:54.803399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:54.817532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 353/691] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:54.818837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:54.824247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86c21c-f5fb-4999-aa7e-4c1eee46014a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:54.829961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:54.897973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73324837-bda3-4829-b325-e18f2431169e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:54.898598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73324837-bda3-4829-b325-e18f2431169e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:54.898872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73324837-bda3-4829-b325-e18f2431169e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:54.899183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73324837-bda3-4829-b325-e18f2431169e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:54.906048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73324837-bda3-4829-b325-e18f2431169e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:54.907523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73324837-bda3-4829-b325-e18f2431169e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-73324837-bda3-4829-b325-e18f2431169e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:54.919109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:54.919987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:54.920967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 339/692] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:54.925967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73324837-bda3-4829-b325-e18f2431169e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:54.954211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d67dd8-ccdd-4559-b8d9-ef4104ec1214 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:55.018300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:55.018922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.027052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.027052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.044352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:55.046196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:55.073882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:55.074862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:55.075806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:55.075806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:55.075806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:55.076136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 354/693] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:55:55.082279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:55.089126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73324837-bda3-4829-b325-e18f2431169e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:55.130611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:55.131704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.131997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.132274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.132972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:55.133280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:55.134718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:55.145997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:55.149235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 340/694] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:55.155512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:55.204536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:55.253485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d92098-2575-425a-ae56-fc0ac1bd9c4a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:55.288888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41b929-43b7-4376-9253-cb53475dae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:55.289492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41b929-43b7-4376-9253-cb53475dae7d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.289770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41b929-43b7-4376-9253-cb53475dae7d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.290031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41b929-43b7-4376-9253-cb53475dae7d None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.290674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41b929-43b7-4376-9253-cb53475dae7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:55.291005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41b929-43b7-4376-9253-cb53475dae7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:55.292479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41b929-43b7-4376-9253-cb53475dae7d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:55.295646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e3cf39-2ae3-4c47-b478-d83ee763b6a1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:55.304797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 355/695] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:55.310033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d179-1ac3-4615-ae62-34b8610c9f7d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.310033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d179-1ac3-4615-ae62-34b8610c9f7d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.310033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d179-1ac3-4615-ae62-34b8610c9f7d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.311497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 356/696] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:55.313018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40d179-1ac3-4615-ae62-34b8610c9f7d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:55.339699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31acd7-6cc0-41d9-9645-df3c2163af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:55.340276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31acd7-6cc0-41d9-9645-df3c2163af38 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.340554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31acd7-6cc0-41d9-9645-df3c2163af38 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.340824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31acd7-6cc0-41d9-9645-df3c2163af38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.341612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31acd7-6cc0-41d9-9645-df3c2163af38 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d4705311aa574da7b9f1b9d31c40cd72) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:55.341887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31acd7-6cc0-41d9-9645-df3c2163af38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:55.343471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31acd7-6cc0-41d9-9645-df3c2163af38 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:55.349710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 341/697] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:55 2022] GET /identity/v3/role_assignments?user.id=d4705311aa574da7b9f1b9d31c40cd72&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:55.350918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5177d-fc59-463d-a635-ed77bea90cc2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.351237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5177d-fc59-463d-a635-ed77bea90cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.351511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5177d-fc59-463d-a635-ed77bea90cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.371470 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-12f5177d-fc59-463d-a635-ed77bea90cc2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:55.372705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-12f5177d-fc59-463d-a635-ed77bea90cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:55.448810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:55.451423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40d179-1ac3-4615-ae62-34b8610c9f7d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:55.452745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 342/698] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:55.471361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:55.490188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:55.490821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.491163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.491482 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.498869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:55.500364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:55.513593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:55.521423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:55.522365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 357/699] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:55:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:55.524132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5177d-fc59-463d-a635-ed77bea90cc2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:55.633934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5177d-fc59-463d-a635-ed77bea90cc2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:55.670161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f8a9d-11f3-48b6-b298-7236e16c9850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:55.670761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8a9d-11f3-48b6-b298-7236e16c9850 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.671067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8a9d-11f3-48b6-b298-7236e16c9850 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.671338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8a9d-11f3-48b6-b298-7236e16c9850 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.672065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f8a9d-11f3-48b6-b298-7236e16c9850 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:55.672367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f8a9d-11f3-48b6-b298-7236e16c9850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:55.673804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f8a9d-11f3-48b6-b298-7236e16c9850 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:55.691359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 343/700] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:55:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:55.696988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:55.742552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0278a-f7b3-4a80-b3ff-6a6f82f3d7a9 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.743201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0278a-f7b3-4a80-b3ff-6a6f82f3d7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.743727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0278a-f7b3-4a80-b3ff-6a6f82f3d7a9 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.745941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 344/701] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:55.746866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0278a-f7b3-4a80-b3ff-6a6f82f3d7a9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:55.811056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:55.841985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181da42-c35d-41a9-8b20-c07157faccb2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:55.861449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0278a-f7b3-4a80-b3ff-6a6f82f3d7a9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:55.887050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd734d-7f2d-4d86-9f9e-21ea739a5366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:55.887714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd734d-7f2d-4d86-9f9e-21ea739a5366 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.887995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd734d-7f2d-4d86-9f9e-21ea739a5366 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.888423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd734d-7f2d-4d86-9f9e-21ea739a5366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.897879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efaa74-b631-4bae-b1b7-4c1d654e24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:55.898678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efaa74-b631-4bae-b1b7-4c1d654e24c2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.899054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efaa74-b631-4bae-b1b7-4c1d654e24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.899408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efaa74-b631-4bae-b1b7-4c1d654e24c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.916175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efaa74-b631-4bae-b1b7-4c1d654e24c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:55.917991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efaa74-b631-4bae-b1b7-4c1d654e24c2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b1efaa74-b631-4bae-b1b7-4c1d654e24c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:55.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:55.936405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:55.937319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:55.937319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:55.937319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:55.937319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 345/702] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:55.938644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c551bf-8214-40ee-af54-b1cc0a5c1a25 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:55.939313 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c551bf-8214-40ee-af54-b1cc0a5c1a25 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:55.939684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c551bf-8214-40ee-af54-b1cc0a5c1a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:55.941726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:55.960663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-75c551bf-8214-40ee-af54-b1cc0a5c1a25 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:55.962038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-75c551bf-8214-40ee-af54-b1cc0a5c1a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:55.969181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd734d-7f2d-4d86-9f9e-21ea739a5366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:55.971196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd734d-7f2d-4d86-9f9e-21ea739a5366 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:56.056606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 346/703] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:56.057568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c551bf-8214-40ee-af54-b1cc0a5c1a25 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:56.107014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 358/704] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:55:56.107414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:56.108202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd734d-7f2d-4d86-9f9e-21ea739a5366 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:56.174246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c551bf-8214-40ee-af54-b1cc0a5c1a25 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:56.212715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:56.213699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.214518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.214794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.215489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:56.215793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:56.217171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:56.230566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 347/705] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:56.235806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:56.308651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:56.335561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd734d-7f2d-4d86-9f9e-21ea739a5366 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:56.371861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ae95-5f2e-4ef2-aa0f-9506f0afddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:56.372634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ae95-5f2e-4ef2-aa0f-9506f0afddc9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.372960 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ae95-5f2e-4ef2-aa0f-9506f0afddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.373233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ae95-5f2e-4ef2-aa0f-9506f0afddc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.376634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:56.383600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2ae95-5f2e-4ef2-aa0f-9506f0afddc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:56.385025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2ae95-5f2e-4ef2-aa0f-9506f0afddc9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:56.386476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912f1c6-3d43-449a-acfe-77a44f93e591 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f5b2ae95-5f2e-4ef2-aa0f-9506f0afddc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:56.400483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:56.401497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:56.402377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:56.402377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:56.402377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:56.402377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 359/706] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:56.403034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c838cd0-88d5-42e4-9812-0373c74edc1b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.403133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c838cd0-88d5-42e4-9812-0373c74edc1b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.403347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c838cd0-88d5-42e4-9812-0373c74edc1b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.406322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 360/707] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:56.407351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c838cd0-88d5-42e4-9812-0373c74edc1b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:56.428591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2884d5-931f-4856-b4e2-e292779b7eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:56.429123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2884d5-931f-4856-b4e2-e292779b7eb2 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.429399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2884d5-931f-4856-b4e2-e292779b7eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.429670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2884d5-931f-4856-b4e2-e292779b7eb2 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.430737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2884d5-931f-4856-b4e2-e292779b7eb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:56.432472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2884d5-931f-4856-b4e2-e292779b7eb2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:56.509783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c838cd0-88d5-42e4-9812-0373c74edc1b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:56.540938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:56.544954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 348/708] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:55:56 2022] POST /identity/v3/users => generated 254 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:55:56.544954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2884d5-931f-4856-b4e2-e292779b7eb2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:56.554859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cf837-0643-4dd3-ae0c-5f370ab68887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:56.555556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cf837-0643-4dd3-ae0c-5f370ab68887 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.555835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cf837-0643-4dd3-ae0c-5f370ab68887 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.556102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cf837-0643-4dd3-ae0c-5f370ab68887 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.556695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cf837-0643-4dd3-ae0c-5f370ab68887 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:56.558083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cf837-0643-4dd3-ae0c-5f370ab68887 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d80cf837-0643-4dd3-ae0c-5f370ab68887 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:56.581005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:56.582102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:56.587993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 361/709] 10.222.0.58 () {56 vars in 1129 bytes} [Tue Aug 23 07:55:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:56.589581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4340e5-60c8-4eca-8b4a-b636754037dc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.589581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4340e5-60c8-4eca-8b4a-b636754037dc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.589581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4340e5-60c8-4eca-8b4a-b636754037dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.606993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9c4340e5-60c8-4eca-8b4a-b636754037dc None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:56.608407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9c4340e5-60c8-4eca-8b4a-b636754037dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:56.679330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2884d5-931f-4856-b4e2-e292779b7eb2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:56.688356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 362/710] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:56.689142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4340e5-60c8-4eca-8b4a-b636754037dc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:56.717790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:56.718741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.719248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.719727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.720601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:56.721079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:56.722693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:56.736685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 349/711] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:56.742560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:56.755145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:56.823944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4340e5-60c8-4eca-8b4a-b636754037dc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:56.853229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:56.856132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49e417-b696-41a3-bf47-b7de4239d0b0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:56.867083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:56.868558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.869095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.869566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.871073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:56.871913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:56.873948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:56.879720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 363/712] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Aug 23 07:55:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:56.884852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:56.896799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:56.897528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:56.897867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:56.898198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:56.905805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:56.907356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:55:56.919107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:56.920254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:56.921419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:55:56.921419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:55:56.921419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:55:56.921419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:55:56.921419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:55:56.921419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:55:56.921419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:55:56.921419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:56.927296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 350/713] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:56.928709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:57.014084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:57.018826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599e228-c506-484f-9a67-c55eae32c489 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:57.054262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:57.055035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:57.055443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:57.055631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38608a9-7a78-4ffe-8c13-99e9b7242451 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:57.055916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:57.056636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:57.057026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:57.059577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:57.066205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 364/714] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Aug 23 07:55:56 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:57.067563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:57.096845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:57.097786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:57.098273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:57.098681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:57.099717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:57.100306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:57.102152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:57.117094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 351/715] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:55:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:57.139886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:57.146332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:57.196397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae462669-ceb5-4b5f-9eff-93cd2494a007 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:57.235993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:57.236637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:57.236911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:57.237178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] PATH_INFO: `/v3/services/network` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:57.237766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:57.239173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:57.249348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:57.250620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:57.251731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 365/716] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:55:57 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:57.257715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:57.349106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca112de2-88b9-4da6-80ac-5d2876da2b0d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:57.395124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:57.400906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4a445-718a-48b4-8cae-93022c621915 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:57.402827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:57.418750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:57.418750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:57.418750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:57.418750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=03cc4435b9b0444b9f278e15398443c7) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:57.418750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:57.418750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:57.419320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 352/717] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:55:57 2022] GET /identity/v3/role_assignments?user.id=03cc4435b9b0444b9f278e15398443c7&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:57.431371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:57.445188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c430470-dc36-4671-95b3-851007823758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:57.445829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c430470-dc36-4671-95b3-851007823758 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:57.446175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c430470-dc36-4671-95b3-851007823758 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:57.446450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c430470-dc36-4671-95b3-851007823758 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:57.447128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c430470-dc36-4671-95b3-851007823758 None admin] RBAC: Adding query filter params (name=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:57.447746 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c430470-dc36-4671-95b3-851007823758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:57.449243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c430470-dc36-4671-95b3-851007823758 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:57.454762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 366/718] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:55:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:57.460224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:57.617439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e952c97-9f5c-4555-abff-a191a71637d7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:57.662061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac71ebb-8f16-4dc3-b25d-b8809f7f535d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:57.662764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac71ebb-8f16-4dc3-b25d-b8809f7f535d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:57.663068 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac71ebb-8f16-4dc3-b25d-b8809f7f535d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:57.663342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac71ebb-8f16-4dc3-b25d-b8809f7f535d None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:57.663971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac71ebb-8f16-4dc3-b25d-b8809f7f535d None admin] RBAC: Adding query filter params (type=network) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:57.664273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac71ebb-8f16-4dc3-b25d-b8809f7f535d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:57.665643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac71ebb-8f16-4dc3-b25d-b8809f7f535d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:57.675962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 353/719] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:55:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:57.785098 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d166-6f54-420c-9cdc-85184aef9773 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:57.785376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d166-6f54-420c-9cdc-85184aef9773 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:57.785649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d166-6f54-420c-9cdc-85184aef9773 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:57.787200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 367/720] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:57.793421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98122b6e-0bec-404d-9f4c-8583f3706cfa None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:57.794022 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98122b6e-0bec-404d-9f4c-8583f3706cfa None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:57.794524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98122b6e-0bec-404d-9f4c-8583f3706cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:57.817288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-98122b6e-0bec-404d-9f4c-8583f3706cfa None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:57.819002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-98122b6e-0bec-404d-9f4c-8583f3706cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:57.884017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:57.902636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 354/721] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:58.311674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504d166-6f54-420c-9cdc-85184aef9773 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:58.428365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504d166-6f54-420c-9cdc-85184aef9773 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:58.470117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae27a14-1288-4a03-becc-c2eaad934015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:58.470745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27a14-1288-4a03-becc-c2eaad934015 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:58.471049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27a14-1288-4a03-becc-c2eaad934015 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:58.471323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27a14-1288-4a03-becc-c2eaad934015 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:58.485926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae27a14-1288-4a03-becc-c2eaad934015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6ae27a14-1288-4a03-becc-c2eaad934015 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:58.491085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:58.492012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:58.492851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 368/722] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:58.499502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98122b6e-0bec-404d-9f4c-8583f3706cfa None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:58.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98122b6e-0bec-404d-9f4c-8583f3706cfa None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:58.701004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:58.702005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:58.702506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:58.703001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:58.703862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:58.704364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:58.705955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:58.719299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 355/723] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:55:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:58.724949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae27a14-1288-4a03-becc-c2eaad934015 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:58.811313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:58.896441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae27a14-1288-4a03-becc-c2eaad934015 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:58.949054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536e810-1d05-41f5-91d9-db7fb8d6ae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:58.950071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e810-1d05-41f5-91d9-db7fb8d6ae12 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:58.950558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e810-1d05-41f5-91d9-db7fb8d6ae12 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:58.951065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536e810-1d05-41f5-91d9-db7fb8d6ae12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:58.966314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536e810-1d05-41f5-91d9-db7fb8d6ae12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:58.968112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536e810-1d05-41f5-91d9-db7fb8d6ae12 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0536e810-1d05-41f5-91d9-db7fb8d6ae12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:55:58.988123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:58.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:55:58.990084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:55:58.990084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:55:58.990084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:58.991885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 369/724] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:58.998453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:59.072854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec51bc-8330-49e2-a0d9-7e004f8418d8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.073549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec51bc-8330-49e2-a0d9-7e004f8418d8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.074030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec51bc-8330-49e2-a0d9-7e004f8418d8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.075894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 370/725] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:55:59.081924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756dc3a-1619-4fd0-90db-9a120ea30997 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.082495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756dc3a-1619-4fd0-90db-9a120ea30997 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.083031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756dc3a-1619-4fd0-90db-9a120ea30997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.098582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:59.105129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c756dc3a-1619-4fd0-90db-9a120ea30997 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:59.109154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c756dc3a-1619-4fd0-90db-9a120ea30997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:59.189425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 371/726] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:59.199246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748aec1-e574-4dba-8d17-580acbbc4ad2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:59.253541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:59.254156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.254433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.254705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.255387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:59.255687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:59.257065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:59.274424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 356/727] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:59.279736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756dc3a-1619-4fd0-90db-9a120ea30997 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:59.319077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:59.428959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756dc3a-1619-4fd0-90db-9a120ea30997 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:59.473451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:59.474012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.474294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.474566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.482533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:59.488116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:55:59.499859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:59.500851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:55:59.501824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:55:59.504226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 372/728] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:55:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:59.505550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:59.510440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:59.612832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b9216-a195-4f17-ad5a-0d38e33ed562 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:59.615322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:59.624106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:59.631542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db496a-ca0a-4b77-af76-47d1ff7b2917 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:59.656258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d215e-4467-42dd-af45-d59c6c92b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:59.656916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d215e-4467-42dd-af45-d59c6c92b25a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.657198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d215e-4467-42dd-af45-d59c6c92b25a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.657471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d215e-4467-42dd-af45-d59c6c92b25a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.672808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d215e-4467-42dd-af45-d59c6c92b25a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:55:59.673191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca01c57-8fac-41ac-9aa7-abd049d5e244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:59.674036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca01c57-8fac-41ac-9aa7-abd049d5e244 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.674541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca01c57-8fac-41ac-9aa7-abd049d5e244 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.675056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca01c57-8fac-41ac-9aa7-abd049d5e244 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.676005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca01c57-8fac-41ac-9aa7-abd049d5e244 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:59.676568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca01c57-8fac-41ac-9aa7-abd049d5e244 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-809d215e-4467-42dd-af45-d59c6c92b25a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:55:59.677518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:55:59.678430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:55:59.679304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 357/729] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:55:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:55:59.685617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688a886-6424-41a1-b48f-fb457cf443c5 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.685617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688a886-6424-41a1-b48f-fb457cf443c5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.685617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688a886-6424-41a1-b48f-fb457cf443c5 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.685617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 358/730] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:59.686897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca01c57-8fac-41ac-9aa7-abd049d5e244 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:59.687835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688a886-6424-41a1-b48f-fb457cf443c5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:59.712550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 373/731] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Aug 23 07:55:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:55:59.714709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac61126-688f-43d9-9430-ce8233bc24f4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.715238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac61126-688f-43d9-9430-ce8233bc24f4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.715741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac61126-688f-43d9-9430-ce8233bc24f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.734134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3ac61126-688f-43d9-9430-ce8233bc24f4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:55:59.735741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3ac61126-688f-43d9-9430-ce8233bc24f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:55:59.810163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 374/732] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:55:59.812169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac61126-688f-43d9-9430-ce8233bc24f4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:59.822747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688a886-6424-41a1-b48f-fb457cf443c5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:55:59.860917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce1413-cc55-40f6-8fee-6e8aa8640f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:55:59.861250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce1413-cc55-40f6-8fee-6e8aa8640f87 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.861417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce1413-cc55-40f6-8fee-6e8aa8640f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.861693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce1413-cc55-40f6-8fee-6e8aa8640f87 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.862356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce1413-cc55-40f6-8fee-6e8aa8640f87 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:55:59.862800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce1413-cc55-40f6-8fee-6e8aa8640f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:55:59.864311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce1413-cc55-40f6-8fee-6e8aa8640f87 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:55:59.873691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 359/733] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:55:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:55:59.875081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2e913-2007-4c50-95f1-42ea18e9d44f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:55:59.875584 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2e913-2007-4c50-95f1-42ea18e9d44f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:55:59.876045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2e913-2007-4c50-95f1-42ea18e9d44f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:55:59.881959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 360/734] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:55:59.881959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2e913-2007-4c50-95f1-42ea18e9d44f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:55:59.913738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:59.916148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:55:59.956219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac61126-688f-43d9-9430-ce8233bc24f4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:00.000873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2e913-2007-4c50-95f1-42ea18e9d44f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:00.006408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484adba-ff32-45a4-90ae-ea77aa9523ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:00.007342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484adba-ff32-45a4-90ae-ea77aa9523ac None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.007852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484adba-ff32-45a4-90ae-ea77aa9523ac None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.008324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484adba-ff32-45a4-90ae-ea77aa9523ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.042835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59435102-8e13-4b48-876f-2f600e01df82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:00.044508 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59435102-8e13-4b48-876f-2f600e01df82 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.045012 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59435102-8e13-4b48-876f-2f600e01df82 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.045794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59435102-8e13-4b48-876f-2f600e01df82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.060637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59435102-8e13-4b48-876f-2f600e01df82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:00.063202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59435102-8e13-4b48-876f-2f600e01df82 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-59435102-8e13-4b48-876f-2f600e01df82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:00.083363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:00.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:00.085245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:00.085245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:00.085245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:00.086695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 361/735] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:00.088045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ee6a-aa02-4175-b7b7-b4c98d7f9b0e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.088475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ee6a-aa02-4175-b7b7-b4c98d7f9b0e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.088957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ee6a-aa02-4175-b7b7-b4c98d7f9b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.091974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484adba-ff32-45a4-90ae-ea77aa9523ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:00.093623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484adba-ff32-45a4-90ae-ea77aa9523ac None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:00.108123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1724ee6a-aa02-4175-b7b7-b4c98d7f9b0e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:00.109411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1724ee6a-aa02-4175-b7b7-b4c98d7f9b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:00.164654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:00.190973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 362/736] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:56:00.190973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ee6a-aa02-4175-b7b7-b4c98d7f9b0e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:00.204254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:00.211272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 375/737] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:55:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 07:56:00.212207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484adba-ff32-45a4-90ae-ea77aa9523ac None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:00.314146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ee6a-aa02-4175-b7b7-b4c98d7f9b0e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:00.323956 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484adba-ff32-45a4-90ae-ea77aa9523ac None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:00.351695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeaa2f0-03a8-4c65-95d0-3292558f06ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:00.352408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeaa2f0-03a8-4c65-95d0-3292558f06ad None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.352692 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeaa2f0-03a8-4c65-95d0-3292558f06ad None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.352992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeaa2f0-03a8-4c65-95d0-3292558f06ad None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.353604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeaa2f0-03a8-4c65-95d0-3292558f06ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:00.355541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeaa2f0-03a8-4c65-95d0-3292558f06ad None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:00.357547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:00.358141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.358455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.358677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.359334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:00.359631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:00.360970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:00.372420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 376/738] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:00.373232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:00.406774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:00.441942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 363/739] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:56:00 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:00.443903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab27f2f-29ad-4595-a14e-2890dafe7a60 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.445153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab27f2f-29ad-4595-a14e-2890dafe7a60 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.445638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab27f2f-29ad-4595-a14e-2890dafe7a60 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.452341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 364/740] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:00.452341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab27f2f-29ad-4595-a14e-2890dafe7a60 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:00.481286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4ee96-c7cb-4f11-8843-eae77aea76ab None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:00.535415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:00.542523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84a304-6d79-4246-99cd-2fddfc0dc00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:00.547189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a304-6d79-4246-99cd-2fddfc0dc00b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.547189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a304-6d79-4246-99cd-2fddfc0dc00b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.547189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84a304-6d79-4246-99cd-2fddfc0dc00b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.557006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84a304-6d79-4246-99cd-2fddfc0dc00b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:00.558555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84a304-6d79-4246-99cd-2fddfc0dc00b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:00.572919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab27f2f-29ad-4595-a14e-2890dafe7a60 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7d84a304-6d79-4246-99cd-2fddfc0dc00b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:00.574833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:00.575999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:00.577158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:00.577158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:00.577158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:00.580810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 377/741] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:00.582416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20e789-f811-4526-a172-35ca0673bef7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.582543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20e789-f811-4526-a172-35ca0673bef7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.582726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20e789-f811-4526-a172-35ca0673bef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.603339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ed20e789-f811-4526-a172-35ca0673bef7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:00.605641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ed20e789-f811-4526-a172-35ca0673bef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:00.616883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:00.617518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.617799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.618069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.628371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:00.629859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:00.645893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:00.646850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 365/742] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:56:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:00.647857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:00.685687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 378/743] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 07:56:00.687241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20e789-f811-4526-a172-35ca0673bef7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:00.704454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:00.764993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a72fc-7b5c-4577-9ec8-c618ffd87fa1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:00.808070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:00.814433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d080334-fc86-47c2-a7f3-69b91ddfb165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:00.815115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d080334-fc86-47c2-a7f3-69b91ddfb165 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.815398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d080334-fc86-47c2-a7f3-69b91ddfb165 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.815676 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d080334-fc86-47c2-a7f3-69b91ddfb165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.826176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20e789-f811-4526-a172-35ca0673bef7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:00.830482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d080334-fc86-47c2-a7f3-69b91ddfb165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-7d080334-fc86-47c2-a7f3-69b91ddfb165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:00.835608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:00.836548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:00.837409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:00.842534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 366/744] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:00.843444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d080334-fc86-47c2-a7f3-69b91ddfb165 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:00.869849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:00.870934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:00.871438 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:00.872115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:00.872953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:00.873456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:00.875210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:00.888795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 379/745] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:00.890049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:00.962471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:00.969597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d080334-fc86-47c2-a7f3-69b91ddfb165 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.001585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b15c28-767a-4f25-8348-6dc2379c9af5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.007849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.008449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.009145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.009419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.010151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:01.010830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.012659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.030330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 367/746] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Aug 23 07:56:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:01.031264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.044567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.045124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.045408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.045684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.046346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:01.046649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.048094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.056981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 380/747] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:56:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:01.057705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.089051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:01.139248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74a6e8-f725-44b4-8bb2-b5249b16fa3e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.165838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbba1f-492e-4734-a981-09f3b4411314 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.175648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.176381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.176658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.176937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.184633 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.186256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.196179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:01.199221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:01.200088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:01.200956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 368/748] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:01.201437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.205326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d74c9-8e76-4f76-862c-ff84c6ce64d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.205935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d74c9-8e76-4f76-862c-ff84c6ce64d6 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.206216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d74c9-8e76-4f76-862c-ff84c6ce64d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.206487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d74c9-8e76-4f76-862c-ff84c6ce64d6 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.210492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d74c9-8e76-4f76-862c-ff84c6ce64d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.210492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d74c9-8e76-4f76-862c-ff84c6ce64d6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.303599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5072e-1c9e-4f44-af58-5f1e820b9098 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.314343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:01.321966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 381/749] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:56:01 2022] POST /identity/v3/services => generated 240 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:01.321966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d74c9-8e76-4f76-862c-ff84c6ce64d6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.354432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4144c-7d8a-414f-8e36-13f39a5d5134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.354432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4144c-7d8a-414f-8e36-13f39a5d5134 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.354432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4144c-7d8a-414f-8e36-13f39a5d5134 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.354432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4144c-7d8a-414f-8e36-13f39a5d5134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.437552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4144c-7d8a-414f-8e36-13f39a5d5134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.440290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4144c-7d8a-414f-8e36-13f39a5d5134 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.465128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d74c9-8e76-4f76-862c-ff84c6ce64d6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.507242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.508524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.508524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.508723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.520430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.522495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:01.537668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:01.538720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:01.539693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:01.539693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:01.539693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:01.539693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 382/750] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:01.539917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.582400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 369/751] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:56:01.584450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4144c-7d8a-414f-8e36-13f39a5d5134 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.595841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:01.596980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:01.658002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383fbf-8e56-4609-a71c-43fb9e45d618 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.699684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.700667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.700973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.701289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.701990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:01.702296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.703920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.720216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4144c-7d8a-414f-8e36-13f39a5d5134 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.723026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 383/752] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:01.724015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.772636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.773833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.774388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.774833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.782337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:01.783345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.784899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.800656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 370/753] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:01.801721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.873238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f86f3-09e6-4307-ac1b-5cb372459c42 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.875658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:01.892425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:01.918763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.919883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.920267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.920656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.936225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.937688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:01.941687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0cfe6-0980-42a8-8458-cef357e02eed None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:01.959592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:01.960628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:01.961548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:01.961548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:01.961548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:01.968560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 384/754] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:01.968560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:01.988495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:01.989750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:01.990234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:01.990709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:01.992230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d4705311aa574da7b9f1b9d31c40cd72) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:01.992752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:01.995463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:02.011042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 371/755] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:01 2022] GET /identity/v3/role_assignments?user.id=d4705311aa574da7b9f1b9d31c40cd72&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:02.011042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:02.142436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51187c-51ec-4840-a5ff-a4261aff7867 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:02.199096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:02.199653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.199928 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.200197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.209236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:02.209929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8095-df0a-409e-a8f6-6c6679960c88 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:02.211167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:02.226478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:02.227515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:02.228544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:02.230829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 385/756] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:56:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:02.236741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:02.241863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:02.249158 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:02.249767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.250082 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.250483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.251248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:02.251605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:02.253087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:02.271859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:02.271996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 372/757] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:02.272764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:02.364202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861698e1-2994-4fa6-9537-6961a47151a6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:02.403988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc891c4-8fb0-4cb5-bef8-172061bed209 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:02.433840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0c6e-e780-4ae7-b938-38b3164d760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:02.435398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0c6e-e780-4ae7-b938-38b3164d760a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.435935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0c6e-e780-4ae7-b938-38b3164d760a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.436404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0c6e-e780-4ae7-b938-38b3164d760a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.438001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca0c6e-e780-4ae7-b938-38b3164d760a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:02.438483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca0c6e-e780-4ae7-b938-38b3164d760a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:02.440703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca0c6e-e780-4ae7-b938-38b3164d760a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:02.445694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:02.446429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.446823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.447224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.455048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:02.456670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:02.464179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 386/758] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:56:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:02.469832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41473-0dba-4de9-b729-20757eaec655 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:02.469988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:02.471037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:02.472182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:02.472182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:02.472182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:02.472182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:02.472182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:02.472182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:02.472182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:02.472182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:02.473107 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41473-0dba-4de9-b729-20757eaec655 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.473293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 373/759] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:02.473675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e41473-0dba-4de9-b729-20757eaec655 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.475514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 387/760] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:02.478685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:02.479152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e41473-0dba-4de9-b729-20757eaec655 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:02.541333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:02.574273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:02.607934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e41473-0dba-4de9-b729-20757eaec655 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:02.617373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bff5bf-a984-4f3c-b07a-d68128236fba None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:02.653189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:02.653807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.654080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.654344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.654709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f85b60-c498-4a60-a5fa-ffc768cf8c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:02.655498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85b60-c498-4a60-a5fa-ffc768cf8c82 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.655947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85b60-c498-4a60-a5fa-ffc768cf8c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.656640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85b60-c498-4a60-a5fa-ffc768cf8c82 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.657906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f85b60-c498-4a60-a5fa-ffc768cf8c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:02.658375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f85b60-c498-4a60-a5fa-ffc768cf8c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:02.659976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f85b60-c498-4a60-a5fa-ffc768cf8c82 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:02.665415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:02.667032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:02.676431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 388/761] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:02.678162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1817f-c655-4eaf-97ad-c554688aefbb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.678635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1817f-c655-4eaf-97ad-c554688aefbb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.679121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1817f-c655-4eaf-97ad-c554688aefbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:02.682163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:02.683679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:02.684569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:02.684569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:02.684569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:02.688367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 374/762] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:02.689196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:02.704251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-52e1817f-c655-4eaf-97ad-c554688aefbb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:02.706051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-52e1817f-c655-4eaf-97ad-c554688aefbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:02.779108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 389/763] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:02.780074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1817f-c655-4eaf-97ad-c554688aefbb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:02.825088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147173d6-13b5-4026-80bb-c846695d3147 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:02.827515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:02.876603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:02.877469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:02.877890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:02.878467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:02.879907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d93e24780ff740b58285575d6efdffa6) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:02.883725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:02.885872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:02.899757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 375/764] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:02 2022] GET /identity/v3/role_assignments?user.id=d93e24780ff740b58285575d6efdffa6&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:02.957147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1817f-c655-4eaf-97ad-c554688aefbb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:02.983347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:03.004466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:03.005097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.005378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.005655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.006446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:03.006748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:03.008156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:03.028658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 390/765] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:03.034369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:03.141834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:03.184893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d16c71-a863-4d63-a8cf-9d27f55f01b4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:03.244898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:03.245481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.245757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.246025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.256345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:03.257787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:03.282199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:03.283209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:03.284105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 376/766] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:03.289907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:03.324447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ff653-6407-4293-b7fb-60aa8165515c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:03.340715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:03.374258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:03.396022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f010942-b7cf-42aa-a6b7-9f4c90a1975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:03.396653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f010942-b7cf-42aa-a6b7-9f4c90a1975b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.396937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f010942-b7cf-42aa-a6b7-9f4c90a1975b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.397194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f010942-b7cf-42aa-a6b7-9f4c90a1975b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.397787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f010942-b7cf-42aa-a6b7-9f4c90a1975b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:03.399378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f010942-b7cf-42aa-a6b7-9f4c90a1975b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0f010942-b7cf-42aa-a6b7-9f4c90a1975b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:03.410258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:03.411289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:03.412130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 391/767] 10.222.0.58 () {56 vars in 1129 bytes} [Tue Aug 23 07:56:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:03.412909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dff74-d62a-462b-a423-a2c39e25c045 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.413472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dff74-d62a-462b-a423-a2c39e25c045 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.413472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dff74-d62a-462b-a423-a2c39e25c045 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.423705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 392/768] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:03.423705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dff74-d62a-462b-a423-a2c39e25c045 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:03.436506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08613f42-c04e-49c8-a9ee-d4b0665d4501 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:03.480642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d253c6-e88b-4856-aa03-75c3781da442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:03.481662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d253c6-e88b-4856-aa03-75c3781da442 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.482212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d253c6-e88b-4856-aa03-75c3781da442 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.482722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d253c6-e88b-4856-aa03-75c3781da442 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.484162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d253c6-e88b-4856-aa03-75c3781da442 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:03.484952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d253c6-e88b-4856-aa03-75c3781da442 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:03.486830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d253c6-e88b-4856-aa03-75c3781da442 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:03.504953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 377/769] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:03.509755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02d12-8b34-44f0-a76c-94aa7abe500d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.509755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02d12-8b34-44f0-a76c-94aa7abe500d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.509755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02d12-8b34-44f0-a76c-94aa7abe500d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.527909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-77e02d12-8b34-44f0-a76c-94aa7abe500d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:03.529424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-77e02d12-8b34-44f0-a76c-94aa7abe500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:03.545065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dff74-d62a-462b-a423-a2c39e25c045 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:03.583057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:03.583749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.584037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.584360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.585092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:03.585399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:03.586915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:03.592864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 393/770] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Aug 23 07:56:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:03.593734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:03.611140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 378/771] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:03.612347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e02d12-8b34-44f0-a76c-94aa7abe500d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:03.647733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:03.652506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:03.701813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9da67-6c4f-4f10-9861-1f472a27cc42 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:03.745352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37c45b-3c22-4db2-bd18-c3ec14ff55f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:03.746255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37c45b-3c22-4db2-bd18-c3ec14ff55f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.747721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37c45b-3c22-4db2-bd18-c3ec14ff55f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.748147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37c45b-3c22-4db2-bd18-c3ec14ff55f7 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/03cc4435b9b0444b9f278e15398443c7/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.752372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37c45b-3c22-4db2-bd18-c3ec14ff55f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=03cc4435b9b0444b9f278e15398443c7, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:03.754041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37c45b-3c22-4db2-bd18-c3ec14ff55f7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:03.790861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e02d12-8b34-44f0-a76c-94aa7abe500d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:03.830218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:03.831845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:03.832332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:03.832755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:03.833593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:03.834066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:03.835686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:03.837700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 394/772] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:56:03 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/03cc4435b9b0444b9f278e15398443c7/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:56:03.842023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 379/773] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Aug 23 07:56:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:03.847921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37c45b-3c22-4db2-bd18-c3ec14ff55f7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:03.993474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37c45b-3c22-4db2-bd18-c3ec14ff55f7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:04.007422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:04.035184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:04.036811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.037136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.037422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.038131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] RBAC: Adding query filter params (interface=public, service_id=83ac1dace1824628a50429061dad6a9e, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:04.038437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:04.039838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:04.048548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 395/774] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Aug 23 07:56:03 2022] GET /identity/v3/endpoints?service_id=83ac1dace1824628a50429061dad6a9e&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:04.053362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:04.054806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:04.157294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:04.169240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236f64e-8ae6-4dc9-b010-e406fed69385 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:04.184971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30867d-de0c-42a8-b20a-a5c3572e1b1b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:04.210778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b7c61-fbd1-4775-8e2c-51d9a4cd19af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:04.214962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7c61-fbd1-4775-8e2c-51d9a4cd19af None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.214962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7c61-fbd1-4775-8e2c-51d9a4cd19af None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.214962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7c61-fbd1-4775-8e2c-51d9a4cd19af None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.214962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b7c61-fbd1-4775-8e2c-51d9a4cd19af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:04.215923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b7c61-fbd1-4775-8e2c-51d9a4cd19af None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:04.222292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 396/775] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:56:04 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:04.229704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb097-d334-4dbc-b13c-2ae7f41ce0da None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.229985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb097-d334-4dbc-b13c-2ae7f41ce0da None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.230265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb097-d334-4dbc-b13c-2ae7f41ce0da None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.236020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 397/776] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:04.237791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2501-9be3-4a4f-88e1-74cbc63cfe47 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.238072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2501-9be3-4a4f-88e1-74cbc63cfe47 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.238338 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2501-9be3-4a4f-88e1-74cbc63cfe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.260271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:04.261075 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.261348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.261660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] PATH_INFO: `/v3/services/network` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.262253 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:04.265446 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:04.267092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-07fc2501-9be3-4a4f-88e1-74cbc63cfe47 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:04.269906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-07fc2501-9be3-4a4f-88e1-74cbc63cfe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:04.286381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:04.287468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:56:04.288338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:04.289088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 380/777] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:04 2022] GET /identity/v3/services/network => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:04.297503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:04.338033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:04.391031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 398/778] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:04.461429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a449-ebf2-48e6-9a6f-fa20e1bf9b8c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:04.507740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:04.508352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.508630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.508904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.509568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:04.509870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:04.511774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:04.518784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 381/779] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:04.524450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc2501-9be3-4a4f-88e1-74cbc63cfe47 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:04.648388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc2501-9be3-4a4f-88e1-74cbc63cfe47 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:04.662209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:04.682028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:04.687491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:04.688097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.688369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.688635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.689263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] RBAC: Adding query filter params (type=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:04.689562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:04.690922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:04.696550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 399/780] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:04.708067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:04.743325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:04.863865 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c4875-91ed-4ae2-ae8e-1a36108401d5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:04.875184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd590726-320e-457c-9d43-a3b643ffaf21 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:04.931285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f553487-4f1b-4f4d-b091-9e1d81f89478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:04.931916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f553487-4f1b-4f4d-b091-9e1d81f89478 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.932188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f553487-4f1b-4f4d-b091-9e1d81f89478 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.932982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f553487-4f1b-4f4d-b091-9e1d81f89478 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.941329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:04.941995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:04.942272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:04.942535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:04.943259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] RBAC: Adding query filter params (interface=public, service_id=23842f05c0ec45599380d490146f42c1, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:04.943618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:04.945223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:04.955500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f553487-4f1b-4f4d-b091-9e1d81f89478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:04.957174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 400/781] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Aug 23 07:56:04 2022] GET /identity/v3/endpoints?service_id=23842f05c0ec45599380d490146f42c1&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0f553487-4f1b-4f4d-b091-9e1d81f89478 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:04.961384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:04.962520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:04.964769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:04.966109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:04.974482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 382/782] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:04.982899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f553487-4f1b-4f4d-b091-9e1d81f89478 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:05.106022 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:05.147322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:05.151919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d3ddb-795e-4f39-a45d-96b981914223 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:05.170417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f553487-4f1b-4f4d-b091-9e1d81f89478 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:05.198665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a18cb-17b7-464d-8074-ae5df65786c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:05.199236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a18cb-17b7-464d-8074-ae5df65786c4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:05.199516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a18cb-17b7-464d-8074-ae5df65786c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:05.199790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a18cb-17b7-464d-8074-ae5df65786c4 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:05.200392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a18cb-17b7-464d-8074-ae5df65786c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:05.201981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a18cb-17b7-464d-8074-ae5df65786c4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:05.207822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 401/783] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:56:04 2022] GET /identity/v3/services => generated 772 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:05.209194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1734-a642-4e42-8f5b-6934787fe88b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:05.209479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1734-a642-4e42-8f5b-6934787fe88b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:05.209759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1734-a642-4e42-8f5b-6934787fe88b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:05.214848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:05.215868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 402/784] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:05.216230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:05.216683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:05.217185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:05.217291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd8e86-613e-4dc6-8cea-3c9534ae62c2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:05.217519 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd8e86-613e-4dc6-8cea-3c9534ae62c2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:05.217797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd8e86-613e-4dc6-8cea-3c9534ae62c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:05.218518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:05.219034 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:05.220684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:05.236168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 383/785] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:56:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:05.238798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:05.250182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9bfd8e86-613e-4dc6-8cea-3c9534ae62c2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:05.251447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9bfd8e86-613e-4dc6-8cea-3c9534ae62c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:05.399428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 403/786] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:05.415526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985416b5-bad6-4752-8f5a-f2ba827f09d7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:05.452205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:05.452826 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:05.453115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:05.453391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:05.455332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:05.464862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:05.466332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:05.481735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:05.482736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:05.483867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:05.483867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:05.483867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:05.483867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 384/787] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:05.490624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd8e86-613e-4dc6-8cea-3c9534ae62c2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:05.594550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:05.610528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd8e86-613e-4dc6-8cea-3c9534ae62c2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:05.655030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:05.655971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:05.656426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:05.656886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:05.657710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:05.658187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:05.659773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:05.675917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 404/788] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:05.681288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:05.681626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:05.814345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d61e2c-846e-49a6-8c72-beaa1323bb75 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:05.859867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f916089-5f7a-4e6f-82f4-be3d4f2f0c97 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:05.881791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33b0e6-0aac-4852-bdb2-df94a5265d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:05.882440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33b0e6-0aac-4852-bdb2-df94a5265d0a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:05.882684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33b0e6-0aac-4852-bdb2-df94a5265d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:05.882934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33b0e6-0aac-4852-bdb2-df94a5265d0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:05.906079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33b0e6-0aac-4852-bdb2-df94a5265d0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0a33b0e6-0aac-4852-bdb2-df94a5265d0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:05.911216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:05.912151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:05.912993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 385/789] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:56:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:05.921701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33b0e6-0aac-4852-bdb2-df94a5265d0a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:05.928832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:05.929463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:05.929736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:05.930000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:05.946304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:05.948526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:05.963441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:05.965016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:05.965944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 405/790] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:05.970210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:05.990752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:05.994853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:06.055535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33b0e6-0aac-4852-bdb2-df94a5265d0a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:06.093262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:06.093878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.094157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.094431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.095124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:06.095428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:06.096798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:06.116776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 386/791] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:56:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:06.124246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:06.203276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bdb21-2a40-4e4c-a8c8-90a5c34f7b5a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:06.272525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:06.273135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.273512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.273794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.274536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:06.274842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:06.276368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:06.313265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 406/792] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Aug 23 07:56:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:06.318750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:06.325901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e398f-8ca1-4c33-8395-992db300cd59 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:06.336310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:06.372632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba147d5f-aee3-42d1-83d3-d6a672e7cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:06.373735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba147d5f-aee3-42d1-83d3-d6a672e7cb6d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.374199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba147d5f-aee3-42d1-83d3-d6a672e7cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.374667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba147d5f-aee3-42d1-83d3-d6a672e7cb6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.386162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba147d5f-aee3-42d1-83d3-d6a672e7cb6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:06.387814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba147d5f-aee3-42d1-83d3-d6a672e7cb6d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:06.398359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-ba147d5f-aee3-42d1-83d3-d6a672e7cb6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:06.404160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:06.405199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:06.406090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:06.406090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:06.406090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:06.406090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 387/793] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:06.407167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f8db8-1f25-4e71-87af-30525d2188a1 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.411799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f8db8-1f25-4e71-87af-30525d2188a1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.411799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f8db8-1f25-4e71-87af-30525d2188a1 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.411799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 388/794] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:06.411799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f8db8-1f25-4e71-87af-30525d2188a1 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:06.442298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ed384-1219-41a9-895f-c8572282ac4b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:06.492459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c84ac8-172d-4659-ade6-f9e0d41f14c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:06.493431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84ac8-172d-4659-ade6-f9e0d41f14c9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.493778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84ac8-172d-4659-ade6-f9e0d41f14c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.494119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c84ac8-172d-4659-ade6-f9e0d41f14c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.507634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c84ac8-172d-4659-ade6-f9e0d41f14c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:06.509762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c84ac8-172d-4659-ade6-f9e0d41f14c9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-83c84ac8-172d-4659-ade6-f9e0d41f14c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:06.530596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:06.531614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:06.532497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:06.532497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:06.532497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:06.532497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 407/795] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:06.534443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb3c8-e5f0-4553-a292-4621b78d02ad None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.535105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb3c8-e5f0-4553-a292-4621b78d02ad None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.535680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb3c8-e5f0-4553-a292-4621b78d02ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.541508 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f8db8-1f25-4e71-87af-30525d2188a1 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:06.566085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-384bb3c8-e5f0-4553-a292-4621b78d02ad None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:06.566085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-384bb3c8-e5f0-4553-a292-4621b78d02ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:06.580917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:06.581531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.583099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.583099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.583099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:06.583375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:06.584637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:06.601451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 389/796] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:06.602374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:06.623239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:06.654564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 408/797] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:06.656522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f207bbc-575f-4549-b6c8-7e8e456ea9d4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.656964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f207bbc-575f-4549-b6c8-7e8e456ea9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.657389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f207bbc-575f-4549-b6c8-7e8e456ea9d4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.659534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 409/798] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:06.660741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f207bbc-575f-4549-b6c8-7e8e456ea9d4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:06.726731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fdb25-2f81-41ec-a6f2-ee82c8a763e8 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:06.740685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:06.782059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113df84b-aad3-475b-8e0b-253ed4780927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:06.782664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113df84b-aad3-475b-8e0b-253ed4780927 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.782971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113df84b-aad3-475b-8e0b-253ed4780927 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.783226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113df84b-aad3-475b-8e0b-253ed4780927 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.783866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113df84b-aad3-475b-8e0b-253ed4780927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:06.784167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113df84b-aad3-475b-8e0b-253ed4780927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:06.785533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113df84b-aad3-475b-8e0b-253ed4780927 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:06.802471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 390/799] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:06.803769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358c61b-04c0-4f0a-835d-d7a242ffe29d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.804396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358c61b-04c0-4f0a-835d-d7a242ffe29d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.804396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358c61b-04c0-4f0a-835d-d7a242ffe29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.821384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f207bbc-575f-4549-b6c8-7e8e456ea9d4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:06.822314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2358c61b-04c0-4f0a-835d-d7a242ffe29d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:06.823844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2358c61b-04c0-4f0a-835d-d7a242ffe29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:06.870380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:06.871645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:06.872326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:06.872943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:06.883930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:06.886228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:06.904258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:06.905226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:06.906165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:06.909422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 410/800] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:56:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:06.910983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:06.934092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 391/801] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:06.935104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358c61b-04c0-4f0a-835d-d7a242ffe29d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:06.946208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:07.048022 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358c61b-04c0-4f0a-835d-d7a242ffe29d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.064245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1649d70-8a0b-4401-bc54-b6030ac4fbca None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.085176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:07.111007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.111631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.111910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.112178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.112968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:07.113276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.114732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.125220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.127062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.127542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.128085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.138420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.139724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 392/802] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Aug 23 07:56:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:07.140681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.140802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:07.159185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:07.160134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:07.161006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:07.161006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:07.161006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:07.161006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:07.161006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:07.161006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:07.161006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:07.161006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:07.167116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 411/803] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:07.168168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.242869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:07.287896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f4f54-db3a-403c-bf24-7375e076490a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.293180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b780a9f-983c-41ae-8bd2-04f68df00733 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.331892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02844496-7a16-4dc2-a077-a4ab935c50e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.332634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844496-7a16-4dc2-a077-a4ab935c50e3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.332918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844496-7a16-4dc2-a077-a4ab935c50e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.333193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844496-7a16-4dc2-a077-a4ab935c50e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.334268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.335145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.335672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.336134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.344666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02844496-7a16-4dc2-a077-a4ab935c50e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:07.349617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-02844496-7a16-4dc2-a077-a4ab935c50e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:07.350761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:07.351696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:07.352507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:07.355321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.357397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 393/804] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:07.358323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02844496-7a16-4dc2-a077-a4ab935c50e3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:07.375837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:07.376812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:07.377709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:07.377709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:07.377709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:07.380687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 412/805] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:07.381773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.417996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:07.475460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02844496-7a16-4dc2-a077-a4ab935c50e3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.497634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8b7bd-33d7-4930-86a8-77ee2243d448 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.502597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:07.515141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.515741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.516016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.516285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.516967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:07.517265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.518688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.536297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 394/806] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:07.537158 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.538029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.538863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.539187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.539516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.540290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.542511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:07.554069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:07.555120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:07.567777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 413/807] 10.222.0.58 () {56 vars in 1129 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:07.568579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.638120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7d71-98a0-43e2-b63a-7cd5422ecb2d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.643592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:07.658931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61e4b-0e46-47b3-a4ba-517aef4772c6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.671748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.672350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.672625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.672901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.673550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:07.673849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.675240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.683159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 395/808] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:07.683943 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.693813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35ef8d-8543-4f55-8720-841dc94f81b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.694415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35ef8d-8543-4f55-8720-841dc94f81b8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.694702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35ef8d-8543-4f55-8720-841dc94f81b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.695006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35ef8d-8543-4f55-8720-841dc94f81b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.695658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35ef8d-8543-4f55-8720-841dc94f81b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:07.695962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35ef8d-8543-4f55-8720-841dc94f81b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.697361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35ef8d-8543-4f55-8720-841dc94f81b8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.709609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 414/809] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:07.710756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a25ff1-9a8d-4994-bfc3-38e3f105b438 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.711071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a25ff1-9a8d-4994-bfc3-38e3f105b438 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.711350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a25ff1-9a8d-4994-bfc3-38e3f105b438 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.712727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 415/810] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:07.715248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a25ff1-9a8d-4994-bfc3-38e3f105b438 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.732679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:07.789071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c596f93-a65d-47fc-9b51-70420b01ef6e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.817431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a25ff1-9a8d-4994-bfc3-38e3f105b438 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:07.821576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e2732-aa37-4b27-a511-a001fd778cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.822220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e2732-aa37-4b27-a511-a001fd778cc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.822739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e2732-aa37-4b27-a511-a001fd778cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.823551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e2732-aa37-4b27-a511-a001fd778cc0 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d4705311aa574da7b9f1b9d31c40cd72/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.825767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e2732-aa37-4b27-a511-a001fd778cc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=d4705311aa574da7b9f1b9d31c40cd72, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.827272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e2732-aa37-4b27-a511-a001fd778cc0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.852095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:07.852686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:07.852959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:07.853223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:07.853831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:07.854154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:07.855536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:07.859810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 416/811] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:07.860557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.872538 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:07.893090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 396/812] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:56:07 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d4705311aa574da7b9f1b9d31c40cd72/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:56:07.894139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e2732-aa37-4b27-a511-a001fd778cc0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:07.967017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dfa86-ed56-4818-9637-dce7f17dadd8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.005959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce16414-f29b-408b-9f4b-ad0282d65ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.006601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce16414-f29b-408b-9f4b-ad0282d65ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.006874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce16414-f29b-408b-9f4b-ad0282d65ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.007167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce16414-f29b-408b-9f4b-ad0282d65ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.018744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce16414-f29b-408b-9f4b-ad0282d65ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.020258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce16414-f29b-408b-9f4b-ad0282d65ff2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.021330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e2732-aa37-4b27-a511-a001fd778cc0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.031855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7ce16414-f29b-408b-9f4b-ad0282d65ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:08.037165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:08.038335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:08.043885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:08.043885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:08.043885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:08.043885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 417/813] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:08.043885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed4646-c8bc-46bc-8221-fc01449cfb8b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.043885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed4646-c8bc-46bc-8221-fc01449cfb8b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.043885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed4646-c8bc-46bc-8221-fc01449cfb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.062842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d4ed4646-c8bc-46bc-8221-fc01449cfb8b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:08.064050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d4ed4646-c8bc-46bc-8221-fc01449cfb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:08.066327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.067081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.067492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.067844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.075509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.077039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:08.089013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:08.089913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 397/814] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:08.091422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:08.129527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:08.140417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 418/815] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:08.141350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed4646-c8bc-46bc-8221-fc01449cfb8b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:08.223968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837172ad-d731-430a-819f-8fb000faf37c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.248376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed4646-c8bc-46bc-8221-fc01449cfb8b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.269629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.270912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.271358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.271887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.272736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:08.273155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.275589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.287426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 398/816] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Aug 23 07:56:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:08.287426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:08.290895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.291756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.292185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.292623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.293415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:08.293882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.295456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.308103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 419/817] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:08.309094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:08.322415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:08.368798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:08.412879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a2430-42b8-4b8c-93d2-176c6071fc4b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.421460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40803d-f4d1-4dd3-a278-5682f9c7a2b0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.455967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.456857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.457277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.457652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.458539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:08.458989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.459422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb7fe4-e6d7-4b0a-9f9a-8eaeae9da4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.460281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb7fe4-e6d7-4b0a-9f9a-8eaeae9da4ba None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.460559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb7fe4-e6d7-4b0a-9f9a-8eaeae9da4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.460831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb7fe4-e6d7-4b0a-9f9a-8eaeae9da4ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.460966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.461490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb7fe4-e6d7-4b0a-9f9a-8eaeae9da4ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.462924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb7fe4-e6d7-4b0a-9f9a-8eaeae9da4ba None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.474690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 399/818] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:08.479079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:08.562769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 420/819] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:56:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:08.564297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb7fe4-e6d7-4b0a-9f9a-8eaeae9da4ba None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:08.576723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:08.596821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7566cf5-cd18-4734-9277-f39debc652d8 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.657313 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.658188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.658572 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.659055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.670535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.672746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.692784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb7fe4-e6d7-4b0a-9f9a-8eaeae9da4ba None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.705352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:08.742066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.742591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.742846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.743274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.746337 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.746337 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:08.761491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:08.762661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:08.763580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 421/820] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:08 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:08.765287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:08.814968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:08.816209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 400/821] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:56:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:08.817152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:08.886077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5392510-1b4f-4338-a9f3-c95a385da6f6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.921470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d004c3-133b-4706-a7fb-c3e33b3255d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.921845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d004c3-133b-4706-a7fb-c3e33b3255d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.922114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d004c3-133b-4706-a7fb-c3e33b3255d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.922391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d004c3-133b-4706-a7fb-c3e33b3255d4 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d93e24780ff740b58285575d6efdffa6/roles/43c2615f91944450952391b6a4dc693c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.938335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d004c3-133b-4706-a7fb-c3e33b3255d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=d93e24780ff740b58285575d6efdffa6, role_id=43c2615f91944450952391b6a4dc693c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:08.939768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d004c3-133b-4706-a7fb-c3e33b3255d4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:08.958747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ebd0-c735-49f2-b9d9-0dfe23790d6e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:08.997168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:08.997953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:08.998215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:08.998555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:08.999301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:08.999675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:09.001121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:09.006236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 401/822] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:09.009378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:09.023129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:09.039013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 422/823] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:56:08 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d93e24780ff740b58285575d6efdffa6/roles/43c2615f91944450952391b6a4dc693c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:56:09.040710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d004c3-133b-4706-a7fb-c3e33b3255d4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:09.103425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:09.213523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d004c3-133b-4706-a7fb-c3e33b3255d4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:09.222483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968b430-3d96-4ddd-b0a4-629d0a4bf7d4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:09.262613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:09.263190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:09.263464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:09.263730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:09.264505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] RBAC: Adding query filter params (type=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:09.265004 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:09.266420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:09.269038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:09.269881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:09.270362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:09.270850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:09.271834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:09.271959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 423/824] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:09.272425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:09.274550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:09.279909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:09.297295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 402/825] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:56:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:09.304446 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:09.373931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:09.386580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:09.389820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269427f-e800-4e7d-b755-71a2ed7c2e20 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:09.423363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce067c04-5718-40b3-b7a8-cdea3e58b98e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:09.427573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb5d41-f7f0-4c37-9854-fb24eae5db3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:09.428367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb5d41-f7f0-4c37-9854-fb24eae5db3a None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:09.428751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb5d41-f7f0-4c37-9854-fb24eae5db3a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:09.429188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb5d41-f7f0-4c37-9854-fb24eae5db3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:09.429954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb5d41-f7f0-4c37-9854-fb24eae5db3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:09.431486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb5d41-f7f0-4c37-9854-fb24eae5db3a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:09.461673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b940d8f-2a4b-44b1-a2f1-dcf1f24edee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:09.462283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b940d8f-2a4b-44b1-a2f1-dcf1f24edee6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:09.465181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b940d8f-2a4b-44b1-a2f1-dcf1f24edee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:09.465181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b940d8f-2a4b-44b1-a2f1-dcf1f24edee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:09.523847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 424/826] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:56:09 2022] POST /identity/v3/endpoints => generated 337 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:09.537724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b940d8f-2a4b-44b1-a2f1-dcf1f24edee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:09.539324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b940d8f-2a4b-44b1-a2f1-dcf1f24edee6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:09.614616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:09.654254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 403/827] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:56:09.661181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb5d41-f7f0-4c37-9854-fb24eae5db3a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:09.862930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed8133-12a8-432d-9e20-bdc47fda778a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:09.863278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed8133-12a8-432d-9e20-bdc47fda778a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:09.863499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed8133-12a8-432d-9e20-bdc47fda778a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:09.865150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 404/828] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:09.870721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e17d2-8d9a-47df-b0f7-12adb7fa670b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:09.871027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e17d2-8d9a-47df-b0f7-12adb7fa670b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:09.871296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e17d2-8d9a-47df-b0f7-12adb7fa670b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:09.893082 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2c3e17d2-8d9a-47df-b0f7-12adb7fa670b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:09.894263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2c3e17d2-8d9a-47df-b0f7-12adb7fa670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:09.943661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb5d41-f7f0-4c37-9854-fb24eae5db3a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:09.979584 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 405/829] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:09.987955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:09.988577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:09.988852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:09.989120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:09.996400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:10.001763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:10.003509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:10.018870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:10.020140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:10.021278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:10.021278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:10.021278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:10.023070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 425/830] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:10.032428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e17d2-8d9a-47df-b0f7-12adb7fa670b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:10.149833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:10.156646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e17d2-8d9a-47df-b0f7-12adb7fa670b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:10.193490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:10.194094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.194370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.194700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.195441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:10.195739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:10.197104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:10.214113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 406/831] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:10.215612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:10.305395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:10.321810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1d42a-c30e-4448-af75-a10458dd9b75 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:10.367763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:10.368917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.370245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.370736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.379420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:10.381858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:10.396864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:10.397883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:10.398763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:10.398763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:10.398763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:10.398763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:10.398763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:10.398763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:10.398763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:10.398763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:10.405285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 426/832] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:10.405691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:10.433441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe4aa-0115-4765-87f4-b2f29b0e7c72 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:10.473162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:10.478648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5653a-81db-4c46-8d0b-0a17be04d93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:10.479638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5653a-81db-4c46-8d0b-0a17be04d93c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.479924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5653a-81db-4c46-8d0b-0a17be04d93c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.484292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5653a-81db-4c46-8d0b-0a17be04d93c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.493140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5653a-81db-4c46-8d0b-0a17be04d93c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cef5653a-81db-4c46-8d0b-0a17be04d93c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:10.500102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:10.501061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:10.501933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:10.502559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 407/833] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:10.508991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5653a-81db-4c46-8d0b-0a17be04d93c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:10.532729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:10.540688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f0a9f-4c31-4a1e-8b9d-0df112e01038 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:10.578757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:10.579416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.579778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.579977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.580636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:10.580939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:10.582588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:10.600483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 427/834] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:10.601802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:10.630868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5653a-81db-4c46-8d0b-0a17be04d93c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:10.669424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:10.670365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.670961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.671458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.672420 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:10.672921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:10.674481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:10.688539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 408/835] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:56:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:10.689711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:10.707990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa577c60-deb4-495e-8f05-1c6d896280e8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:10.718642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:10.742835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea3c6b-2c36-4b8e-8af6-6935d14eb564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:10.743599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3c6b-2c36-4b8e-8af6-6935d14eb564 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.743868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3c6b-2c36-4b8e-8af6-6935d14eb564 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.744185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3c6b-2c36-4b8e-8af6-6935d14eb564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.744969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea3c6b-2c36-4b8e-8af6-6935d14eb564 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=03cc4435b9b0444b9f278e15398443c7) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:10.745272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea3c6b-2c36-4b8e-8af6-6935d14eb564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:10.746690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea3c6b-2c36-4b8e-8af6-6935d14eb564 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:10.752992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 428/836] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:10 2022] GET /identity/v3/role_assignments?user.id=03cc4435b9b0444b9f278e15398443c7&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:10.754233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bcc35-cc2f-4fdf-b0ec-39db10699b8f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.754886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bcc35-cc2f-4fdf-b0ec-39db10699b8f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.755980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bcc35-cc2f-4fdf-b0ec-39db10699b8f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.757999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 429/837] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:10.764605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f910-2101-4db5-94b2-9e0b514481dd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.764915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f910-2101-4db5-94b2-9e0b514481dd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.765152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f910-2101-4db5-94b2-9e0b514481dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.787023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8f22f910-2101-4db5-94b2-9e0b514481dd None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:10.788633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8f22f910-2101-4db5-94b2-9e0b514481dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:10.800275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:10.820585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c09e3-c5ab-44db-ad7f-60e678de2a9f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:10.863454 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:10.864146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:10.864425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:10.864692 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:10.883208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:10.885110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:10.891452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 430/838] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:10.902245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:10.903512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:10.904449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:10.904449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:10.904449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:10.906519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 409/839] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:10.912529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22f910-2101-4db5-94b2-9e0b514481dd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.015907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22f910-2101-4db5-94b2-9e0b514481dd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.018670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:11.052713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:11.053320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.053595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.053863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.054502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:11.054800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:11.056244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:11.068973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 431/840] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:11.074716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.152749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.182988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495f847-8c24-4a50-812f-cc0f79ae1918 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.215670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:11.221136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:11.221760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.229055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.229055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.230396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:11.231984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:11.245157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:11.246149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:11.247334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 410/841] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:11.252361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.263639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ba18e-b274-465b-8228-a714fbd25661 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.306976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:11.307881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.308167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.308443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.309031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:11.310571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:11.326964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:11.327991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:11.328822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 432/842] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:11.331639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.365318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:11.372569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff8df1-ab17-482a-a1e7-928aacf134f5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.408296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf348ca3-74e6-41d8-92dd-352f573e944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:11.408893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf348ca3-74e6-41d8-92dd-352f573e944c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.409167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf348ca3-74e6-41d8-92dd-352f573e944c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.409436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf348ca3-74e6-41d8-92dd-352f573e944c None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.410152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf348ca3-74e6-41d8-92dd-352f573e944c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:11.410472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf348ca3-74e6-41d8-92dd-352f573e944c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:11.411901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf348ca3-74e6-41d8-92dd-352f573e944c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:11.428901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 411/843] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Aug 23 07:56:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:11.430140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299126-49d7-4c5d-90c6-c9b2a478a043 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.430428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299126-49d7-4c5d-90c6-c9b2a478a043 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.430704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299126-49d7-4c5d-90c6-c9b2a478a043 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.432073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 412/844] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:11.434606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299126-49d7-4c5d-90c6-c9b2a478a043 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.441083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64742ad-9abb-4ce5-8598-7c973c40c53a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.477167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071dc63-3d78-4d94-9190-be21c239f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:11.477799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dc63-3d78-4d94-9190-be21c239f268 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.478080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dc63-3d78-4d94-9190-be21c239f268 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.478351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dc63-3d78-4d94-9190-be21c239f268 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.479035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071dc63-3d78-4d94-9190-be21c239f268 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:11.479341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071dc63-3d78-4d94-9190-be21c239f268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:11.480743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071dc63-3d78-4d94-9190-be21c239f268 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:11.485342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 433/845] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:11.486518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4335b62-1211-472b-8a2e-d1d81d8a56a7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.486799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4335b62-1211-472b-8a2e-d1d81d8a56a7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.487102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4335b62-1211-472b-8a2e-d1d81d8a56a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.494096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:11.505262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c4335b62-1211-472b-8a2e-d1d81d8a56a7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:11.506594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c4335b62-1211-472b-8a2e-d1d81d8a56a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:11.547419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299126-49d7-4c5d-90c6-c9b2a478a043 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.580324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 434/846] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:11.584410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e256ced-1807-4d66-97b6-cf2d71d0b706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:11.585256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e256ced-1807-4d66-97b6-cf2d71d0b706 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.585723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e256ced-1807-4d66-97b6-cf2d71d0b706 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.586154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e256ced-1807-4d66-97b6-cf2d71d0b706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.587878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4335b62-1211-472b-8a2e-d1d81d8a56a7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.618247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:11.665348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e256ced-1807-4d66-97b6-cf2d71d0b706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:11.668114 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e256ced-1807-4d66-97b6-cf2d71d0b706 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:11.720946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:11.721139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4335b62-1211-472b-8a2e-d1d81d8a56a7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.768403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:11.769428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.769713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.769987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.770898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:11.771243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:11.772887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:11.780050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 435/847] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:11.793060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 413/848] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:56:11.794212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.849986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e256ced-1807-4d66-97b6-cf2d71d0b706 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:11.914800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:11.927415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff94f9-d5b6-49af-a988-6ac3bb9e3d14 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.969285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:11.969921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:11.970202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:11.970473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:11.973745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e256ced-1807-4d66-97b6-cf2d71d0b706 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:11.983440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:11.984969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:12.001759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:12.002766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:12.003707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:12.003707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:12.003707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:12.003707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 436/849] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:12.008667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.021299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b2bf-9a25-41f9-848c-9b080c82909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.022209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b2bf-9a25-41f9-848c-9b080c82909c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.023163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b2bf-9a25-41f9-848c-9b080c82909c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.023655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2b2bf-9a25-41f9-848c-9b080c82909c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.037180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2b2bf-9a25-41f9-848c-9b080c82909c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cda2b2bf-9a25-41f9-848c-9b080c82909c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:12.044813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:12.045728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:56:12.046559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:12.052977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 414/850] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:56:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:12.055169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b2bf-9a25-41f9-848c-9b080c82909c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.071319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:12.133661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc697c-a521-4e04-bf78-224207af203b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:12.178225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.179109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.179770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.180669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.181799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:12.182313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:12.183751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2b2bf-9a25-41f9-848c-9b080c82909c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:12.184453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.194147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:12.202317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 437/851] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:12.207402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.222517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.224554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.225000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.225558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.226592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:12.227221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:12.228802 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.239904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 415/852] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:12.246065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.336044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:12.341625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95839d7-7fb8-486b-92e6-6d8c1caa2b25 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:12.372224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ab34-631c-407e-9c57-4d05db9e5ddc None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:12.387539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.388112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.388664 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.388947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.397902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:12.399930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:12.413192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:12.414118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:12.415031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:12.415031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:12.415031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:12.415031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:12.415031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:12.415031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:12.415031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:12.415031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:12.417476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.417725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 438/853] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:12.418057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.418441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.418712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.425536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.430924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:12.433120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:12.449991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:12.451232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:12.452113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:12.452113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:12.452113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:12.452113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 416/854] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:12.456700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.492594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:12.536869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b29faf-f17c-4bd7-9367-9aba78f37769 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:12.587146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.587757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.588037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.588905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.589605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:12.589816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f5b81-4a9b-4f22-9dc0-6c386512db15 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:12.589999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:12.591474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.609469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 439/855] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:12.615424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.618516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:12.636443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.637119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.637396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.637771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.638615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:12.638993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:12.640809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.661990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 417/856] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:12.663909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.738766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e77f5-7463-4546-a3c3-dc8699c8ac92 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:12.768396 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:12.778617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c216e2e-80b0-4b73-a7dc-5ed57e2e771c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:12.797149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ced7b-5155-43ca-a9cf-cca241d12c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.797754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ced7b-5155-43ca-a9cf-cca241d12c3c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.798302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ced7b-5155-43ca-a9cf-cca241d12c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.798592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ced7b-5155-43ca-a9cf-cca241d12c3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.799358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ced7b-5155-43ca-a9cf-cca241d12c3c None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d4705311aa574da7b9f1b9d31c40cd72) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:12.799761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ced7b-5155-43ca-a9cf-cca241d12c3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:12.801692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ced7b-5155-43ca-a9cf-cca241d12c3c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.811504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 440/857] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/role_assignments?user.id=d4705311aa574da7b9f1b9d31c40cd72&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:12.825212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:12.826262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.826759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.827362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.836124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:12.837983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:12.844124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cede04-d68c-41a5-8246-b999b31e00e8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.844443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cede04-d68c-41a5-8246-b999b31e00e8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.844668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cede04-d68c-41a5-8246-b999b31e00e8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.846189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 441/858] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:12.852166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6f90e-ac53-4167-81e4-fa593df2b46b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:12.852776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6f90e-ac53-4167-81e4-fa593df2b46b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:12.853278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6f90e-ac53-4167-81e4-fa593df2b46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:12.855678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:12.856616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:12.857600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 418/859] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:12.862857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:12.878524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-62a6f90e-ac53-4167-81e4-fa593df2b46b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:12.880292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-62a6f90e-ac53-4167-81e4-fa593df2b46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:12.919318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:12.978991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 442/860] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:12.993975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b497b32-ad89-47c6-9d7f-c1830de886d1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:13.029623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:13.030233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.030514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.030789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.031548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:13.031857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:13.033312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:13.051841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 419/861] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Aug 23 07:56:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:13.057308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6f90e-ac53-4167-81e4-fa593df2b46b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:13.157497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6f90e-ac53-4167-81e4-fa593df2b46b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:13.171550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:13.193421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e8d2c-279d-4615-928b-8d01273b2bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:13.194410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e8d2c-279d-4615-928b-8d01273b2bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.194903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e8d2c-279d-4615-928b-8d01273b2bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.195427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e8d2c-279d-4615-928b-8d01273b2bc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.257082 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:13.269544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e8d2c-279d-4615-928b-8d01273b2bc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:13.271450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e8d2c-279d-4615-928b-8d01273b2bc1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:13.382317 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1b7cd-c132-4f52-9eac-ccf06f2b94f6 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:13.383665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:13.396498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 443/862] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:56:13.402238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e8d2c-279d-4615-928b-8d01273b2bc1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:13.421086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:13.423418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:13.424332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.424903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.425376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.426159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:13.427868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:13.437639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:13.438571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:13.439809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:13.442245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 420/863] 10.222.0.58 () {56 vars in 1132 bytes} [Tue Aug 23 07:56:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:13.448600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:13.514348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e8d2c-279d-4615-928b-8d01273b2bc1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:13.549995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:13.551058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.551532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.551968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.558151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70487018-decd-43cd-a44e-858c75daa6ec None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:13.563155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:13.564826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:13.580734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:13.581688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:13.582566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:13.582566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:13.582566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:13.589628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 444/864] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:13.590762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:13.593330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:13.594048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.594333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.594606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.596250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:13.596250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:13.597169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:13.602151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 421/865] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:13.607977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:13.643287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:13.657542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:13.695496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdfc42-654f-4507-a6d8-06c907f0aa84 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:13.718442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a2ab0-4e67-4eae-98b2-a27b5b69525e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:13.739592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cc665-96f2-420f-a5b3-19c2abed5476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:13.740469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc665-96f2-420f-a5b3-19c2abed5476 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.740744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc665-96f2-420f-a5b3-19c2abed5476 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.741013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc665-96f2-420f-a5b3-19c2abed5476 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.741784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc665-96f2-420f-a5b3-19c2abed5476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:13.742380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc665-96f2-420f-a5b3-19c2abed5476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:13.743767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc665-96f2-420f-a5b3-19c2abed5476 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:13.758604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 445/866] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:13.758604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63148799-1c20-4e36-b859-b0f6171be36e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.758897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63148799-1c20-4e36-b859-b0f6171be36e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.759018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63148799-1c20-4e36-b859-b0f6171be36e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.763407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 446/867] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:13.764675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63148799-1c20-4e36-b859-b0f6171be36e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:13.765796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f36494-8e07-4366-9dfb-13b089df525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:13.766408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f36494-8e07-4366-9dfb-13b089df525d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.766687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f36494-8e07-4366-9dfb-13b089df525d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.766985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f36494-8e07-4366-9dfb-13b089df525d None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.767619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f36494-8e07-4366-9dfb-13b089df525d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:13.767919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f36494-8e07-4366-9dfb-13b089df525d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:13.769286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f36494-8e07-4366-9dfb-13b089df525d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:13.774324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 422/868] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:13 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:13.775445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03577964-f016-4288-a922-53185cab4cc6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.776071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03577964-f016-4288-a922-53185cab4cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.776540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03577964-f016-4288-a922-53185cab4cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.797533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-03577964-f016-4288-a922-53185cab4cc6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:13.798762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-03577964-f016-4288-a922-53185cab4cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:13.882580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 423/869] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:13.906451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63148799-1c20-4e36-b859-b0f6171be36e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:13.943346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:13.952849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:13.952849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:13.952849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:13.952849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:13.958774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:13.960429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:13.973327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:13.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:13.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:13.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:13.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:13.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:13.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:13.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:13.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:13.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:13.977968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 447/870] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:13.984499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03577964-f016-4288-a922-53185cab4cc6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:14.013610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:14.092848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03577964-f016-4288-a922-53185cab4cc6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:14.128939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:14.129541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:14.129834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:14.130089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:14.130772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:14.131095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:14.132507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:14.144781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 424/871] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:14.151374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:14.154091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:14.248032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:14.264947 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6db51-1944-4e38-bd94-d953139884c4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:14.281717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2706e-de93-436b-b4f7-fa52937ec8bd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:14.311554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103187d-8cc6-4bfb-819c-45f6bd857095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:14.312750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103187d-8cc6-4bfb-819c-45f6bd857095 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:14.313596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103187d-8cc6-4bfb-819c-45f6bd857095 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:14.314094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103187d-8cc6-4bfb-819c-45f6bd857095 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:14.315186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103187d-8cc6-4bfb-819c-45f6bd857095 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:14.316745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103187d-8cc6-4bfb-819c-45f6bd857095 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:14.318551 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d7c18-a7a1-40a6-9891-2ba36778f94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:14.319220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7c18-a7a1-40a6-9891-2ba36778f94e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:14.319501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7c18-a7a1-40a6-9891-2ba36778f94e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:14.319839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7c18-a7a1-40a6-9891-2ba36778f94e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:14.320571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d7c18-a7a1-40a6-9891-2ba36778f94e None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d93e24780ff740b58285575d6efdffa6) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:14.320877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d7c18-a7a1-40a6-9891-2ba36778f94e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:14.322381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d7c18-a7a1-40a6-9891-2ba36778f94e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:14.329664 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 448/872] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:14 2022] GET /identity/v3/role_assignments?user.id=d93e24780ff740b58285575d6efdffa6&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:14.402828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 425/873] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:56:14 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:56:14.842176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e77e-c893-4030-9eaa-2ad16fd8fd71 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:14.842683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e77e-c893-4030-9eaa-2ad16fd8fd71 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:14.842875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e77e-c893-4030-9eaa-2ad16fd8fd71 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:14.844343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 449/874] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:14.849889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cae19-c7de-4a07-b553-d2808c5cc688 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:14.850416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cae19-c7de-4a07-b553-d2808c5cc688 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:14.850896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cae19-c7de-4a07-b553-d2808c5cc688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:14.870525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6f8cae19-c7de-4a07-b553-d2808c5cc688 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:14.872112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6f8cae19-c7de-4a07-b553-d2808c5cc688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:14.949603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 426/875] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:15.214878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236e77e-c893-4030-9eaa-2ad16fd8fd71 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:15.265901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:15.319217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236e77e-c893-4030-9eaa-2ad16fd8fd71 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:15.353209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:15.354356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:15.354633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:15.354894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:15.355490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:15.356842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:15.365619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:15.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:15.368059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:15.371093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 450/876] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:15.373092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cae19-c7de-4a07-b553-d2808c5cc688 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:15.447157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:15.480435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cae19-c7de-4a07-b553-d2808c5cc688 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:15.516698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86ba19-0d04-405f-9485-3d963840a345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:15.517661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86ba19-0d04-405f-9485-3d963840a345 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:15.518179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86ba19-0d04-405f-9485-3d963840a345 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:15.518655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86ba19-0d04-405f-9485-3d963840a345 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:15.519547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86ba19-0d04-405f-9485-3d963840a345 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:15.520057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86ba19-0d04-405f-9485-3d963840a345 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:15.521637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86ba19-0d04-405f-9485-3d963840a345 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:15.528048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 427/877] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:15.536296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:15.637651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612c15d-f49a-41f6-acd0-9c3518549d11 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:15.650497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:15.671451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9f4e3-9560-4d24-a42f-2870e9d3cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:15.672049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9f4e3-9560-4d24-a42f-2870e9d3cb93 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:15.672327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9f4e3-9560-4d24-a42f-2870e9d3cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:15.672588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9f4e3-9560-4d24-a42f-2870e9d3cb93 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:15.673204 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9f4e3-9560-4d24-a42f-2870e9d3cb93 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:15.673501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9f4e3-9560-4d24-a42f-2870e9d3cb93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:15.674853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9f4e3-9560-4d24-a42f-2870e9d3cb93 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:15.679524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 451/878] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:15.794035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11c238-ac17-463b-8369-7cefbaa1066b None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:15.794356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11c238-ac17-463b-8369-7cefbaa1066b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:15.794586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11c238-ac17-463b-8369-7cefbaa1066b None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:15.796237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 428/879] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:15.802081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb893e-749f-44b5-9907-818a31ee9a53 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:15.802363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb893e-749f-44b5-9907-818a31ee9a53 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:15.802641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb893e-749f-44b5-9907-818a31ee9a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:15.821688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5ceb893e-749f-44b5-9907-818a31ee9a53 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:15.823449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5ceb893e-749f-44b5-9907-818a31ee9a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:15.896482 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 452/880] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:16.168804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11c238-ac17-463b-8369-7cefbaa1066b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:16.271980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11c238-ac17-463b-8369-7cefbaa1066b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:16.307917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bde41-dbc3-45d6-b145-2b2b53a95eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:16.308933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bde41-dbc3-45d6-b145-2b2b53a95eea None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:16.309415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bde41-dbc3-45d6-b145-2b2b53a95eea None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:16.309897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bde41-dbc3-45d6-b145-2b2b53a95eea None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:16.310785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bde41-dbc3-45d6-b145-2b2b53a95eea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:16.312376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bde41-dbc3-45d6-b145-2b2b53a95eea None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:16.316013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:16.407622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8162f93-ce6c-4de4-b907-dd98c5af3248 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:16.407936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8162f93-ce6c-4de4-b907-dd98c5af3248 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:16.408207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8162f93-ce6c-4de4-b907-dd98c5af3248 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:16.409704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 453/881] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:16.414425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddc27-8872-4290-8254-c0c096cda718 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:16.414710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddc27-8872-4290-8254-c0c096cda718 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:16.415011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeddc27-8872-4290-8254-c0c096cda718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:16.433542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-caeddc27-8872-4290-8254-c0c096cda718 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:16.434684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-caeddc27-8872-4290-8254-c0c096cda718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:16.438705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 429/882] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:56:16 2022] POST /identity/v3/services => generated 269 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:16.440800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dcab1-7559-4ef5-8b58-c8c5e9dac654 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:16.441371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dcab1-7559-4ef5-8b58-c8c5e9dac654 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:16.441857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dcab1-7559-4ef5-8b58-c8c5e9dac654 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:16.444651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 430/883] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:16.450715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b0dc-caf0-4c09-b10c-dd821ffa0064 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:16.451800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b0dc-caf0-4c09-b10c-dd821ffa0064 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:16.452488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b0dc-caf0-4c09-b10c-dd821ffa0064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:16.480891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3519b0dc-caf0-4c09-b10c-dd821ffa0064 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:16.482380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3519b0dc-caf0-4c09-b10c-dd821ffa0064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:16.531398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 454/884] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:16.574124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 431/885] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:16.810665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeddc27-8872-4290-8254-c0c096cda718 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:16.843374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:16.850128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b0dc-caf0-4c09-b10c-dd821ffa0064 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:16.935555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeddc27-8872-4290-8254-c0c096cda718 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:16.964013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b0dc-caf0-4c09-b10c-dd821ffa0064 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:16.972557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf37e-5bbb-4647-b993-62d4a1fb8c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:16.972888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf37e-5bbb-4647-b993-62d4a1fb8c21 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:16.973175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf37e-5bbb-4647-b993-62d4a1fb8c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:16.973444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf37e-5bbb-4647-b993-62d4a1fb8c21 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:16.984245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdf37e-5bbb-4647-b993-62d4a1fb8c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-56cdf37e-5bbb-4647-b993-62d4a1fb8c21 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:16.989007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:16.989945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:16.990772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 455/886] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Aug 23 07:56:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:16.996131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf37e-5bbb-4647-b993-62d4a1fb8c21 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.002169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:17.013640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.014275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.014555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.016181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.016181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.016790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:17.025468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:17.027018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:17.028257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:17.031088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 432/887] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:17.033401 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.112306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:17.122790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf37e-5bbb-4647-b993-62d4a1fb8c21 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.145992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258d794-d5e8-48be-979e-e2c50e9cefd0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.160580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.161413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.161843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.162261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.163269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:17.163719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.165244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.178588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 456/888] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Aug 23 07:56:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:17.179692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.183428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.184215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.184686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.185183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.186009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:17.186527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.188124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.193600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 433/889] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:17.199662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.264210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:17.285935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c22ebf-0043-4418-8041-89a89e64588f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.321284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.322140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.323126 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.323548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.323684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31651b4-018b-451c-b41d-2559070f30a0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.335102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.336748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:17.351913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:17.352995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:17.353937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:17.353937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:17.353937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:17.355977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 457/890] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:17.361709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.371193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.372297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.372661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.373090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.373808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:17.374206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.375286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:17.376160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.389161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 434/891] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:17.390537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.476513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabadae-67dc-4e54-b060-6c26d248e00e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.517984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.518911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.519444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.519898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.520684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:17.521134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.522705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.525617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42655026-9b6d-4f48-bfa1-ba7686a9b482 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.540722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 458/892] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:17.542351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.545121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:17.567176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a208ee-8040-48cd-8167-8f2ae098071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.568116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a208ee-8040-48cd-8167-8f2ae098071f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.568551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a208ee-8040-48cd-8167-8f2ae098071f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.568983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a208ee-8040-48cd-8167-8f2ae098071f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.569836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a208ee-8040-48cd-8167-8f2ae098071f None admin] RBAC: Adding query filter params (interface=public, service_id=0683b8bf42b146609abedfa614b958f5, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:17.570343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a208ee-8040-48cd-8167-8f2ae098071f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.571906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a208ee-8040-48cd-8167-8f2ae098071f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.585245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 435/893] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/endpoints?service_id=0683b8bf42b146609abedfa614b958f5&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:17.586880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a0e7-3ae1-4435-96c7-8b38aee80b1e None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.587422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a0e7-3ae1-4435-96c7-8b38aee80b1e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.587730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616a0e7-3ae1-4435-96c7-8b38aee80b1e None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.589581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 436/894] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:17.590890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616a0e7-3ae1-4435-96c7-8b38aee80b1e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.616603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:17.657072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd056e-b18d-457a-8597-84ea574d93a7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.692552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22daa76e-f12a-4e26-a16e-81cda09946d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.693178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daa76e-f12a-4e26-a16e-81cda09946d0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.693455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daa76e-f12a-4e26-a16e-81cda09946d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.693728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daa76e-f12a-4e26-a16e-81cda09946d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.696982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616a0e7-3ae1-4435-96c7-8b38aee80b1e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.701155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daa76e-f12a-4e26-a16e-81cda09946d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.702598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daa76e-f12a-4e26-a16e-81cda09946d0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-22daa76e-f12a-4e26-a16e-81cda09946d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:17.714455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:17.715437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:17.716395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 459/895] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:17.716977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f98b09-f313-41b3-862c-0ad036452805 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.716977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f98b09-f313-41b3-862c-0ad036452805 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.717652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f98b09-f313-41b3-862c-0ad036452805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.734829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-89f98b09-f313-41b3-862c-0ad036452805 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:17.736042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-89f98b09-f313-41b3-862c-0ad036452805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:17.737789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc39f5c-df48-48f0-91f4-1afe9bf95cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.738380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc39f5c-df48-48f0-91f4-1afe9bf95cda None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.738660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc39f5c-df48-48f0-91f4-1afe9bf95cda None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.738935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc39f5c-df48-48f0-91f4-1afe9bf95cda None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.739569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc39f5c-df48-48f0-91f4-1afe9bf95cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.740935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc39f5c-df48-48f0-91f4-1afe9bf95cda None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.751035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 437/896] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:56:17.751035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc39f5c-df48-48f0-91f4-1afe9bf95cda None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.805094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:17.843736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 460/897] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:17.872182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc39f5c-df48-48f0-91f4-1afe9bf95cda None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:17.911283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:17.911894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:17.912173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:17.912445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:17.913182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:17.913486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:17.914916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:17.933388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 438/898] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:17.939069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f98b09-f313-41b3-862c-0ad036452805 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:17.998556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:18.043541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f98b09-f313-41b3-862c-0ad036452805 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:18.079722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f253d-c13a-48aa-b09a-65b47b9817dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:18.080253 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f253d-c13a-48aa-b09a-65b47b9817dd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:18.080530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f253d-c13a-48aa-b09a-65b47b9817dd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:18.080797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f253d-c13a-48aa-b09a-65b47b9817dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:18.121706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:18.151635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f253d-c13a-48aa-b09a-65b47b9817dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:18.153100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f253d-c13a-48aa-b09a-65b47b9817dd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:18.209250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:18.234875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ad57b-0559-4881-b110-e3b973ca7dc1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:18.236782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:18.259429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 461/899] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:17 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:56:18.261070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f253d-c13a-48aa-b09a-65b47b9817dd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:18.282134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0feee-2abe-4fa3-b754-e8915169a67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:18.283224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0feee-2abe-4fa3-b754-e8915169a67c None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:18.283861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0feee-2abe-4fa3-b754-e8915169a67c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:18.284427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0feee-2abe-4fa3-b754-e8915169a67c None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:18.285294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0feee-2abe-4fa3-b754-e8915169a67c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:18.286982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0feee-2abe-4fa3-b754-e8915169a67c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:18.365961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 439/900] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:56:18 2022] POST /identity/v3/services => generated 241 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:18.370038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f253d-c13a-48aa-b09a-65b47b9817dd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:18.424510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41922f57-47f9-4205-b76d-110f2d1c7f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:18.425643 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41922f57-47f9-4205-b76d-110f2d1c7f55 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:18.426127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41922f57-47f9-4205-b76d-110f2d1c7f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:18.426617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41922f57-47f9-4205-b76d-110f2d1c7f55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:18.439214 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41922f57-47f9-4205-b76d-110f2d1c7f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:18.441074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41922f57-47f9-4205-b76d-110f2d1c7f55 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-41922f57-47f9-4205-b76d-110f2d1c7f55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:18.458925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:18.459959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:18.460848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:18.460848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:18.460848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:18.467369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 462/901] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:18.469876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0feee-2abe-4fa3-b754-e8915169a67c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:18.474038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891cfc-ad00-4d68-a238-7c19ad46a0b3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:18.474312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891cfc-ad00-4d68-a238-7c19ad46a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:18.474579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891cfc-ad00-4d68-a238-7c19ad46a0b3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:18.476269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 463/902] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:18.482531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021752cc-a801-45b7-9231-61857afd1a6b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:18.482744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021752cc-a801-45b7-9231-61857afd1a6b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:18.482983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021752cc-a801-45b7-9231-61857afd1a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:18.491120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:18.502264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-021752cc-a801-45b7-9231-61857afd1a6b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:18.503522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-021752cc-a801-45b7-9231-61857afd1a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:18.584667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 464/903] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:18.600354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0feee-2abe-4fa3-b754-e8915169a67c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:18.640212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:18.640884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:18.641229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:18.641617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:18.642327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:18.642718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:18.644170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:18.657036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 440/904] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:18.662668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021752cc-a801-45b7-9231-61857afd1a6b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:18.768689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021752cc-a801-45b7-9231-61857afd1a6b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:18.804236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:18.804864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:18.805140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:18.805409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:18.807427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:18.815496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:18.816971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:18.829529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:18.830469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:18.831545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:18.831545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:18.831545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:18.831545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:18.831545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:18.831545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:18.831545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:18.831545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:18.835050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 465/905] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:18.836765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:18.854172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:18.864629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:18.961388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba0b00-18c1-470a-a557-93d8bda0ea98 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:18.972746 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f945885-0ded-4b78-97e5-f7712b372b2c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:19.003605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:19.004933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.005659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.006143 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.007533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:19.008320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:19.010791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:19.012979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:19.013605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.013905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.014507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.015117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:19.016464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:19.025182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 441/906] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:19.026677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:19.027825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:19.028741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:19.030833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:19.032555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 466/907] 10.222.0.58 () {56 vars in 1132 bytes} [Tue Aug 23 07:56:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:19.033366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:19.099750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:19.122923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:19.142374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e4c8-566e-4193-99b7-48bbd57a9dae None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:19.147293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f79ecd-57f7-4fec-a398-526b303ae7f0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:19.182752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:19.183377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.183639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.183901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.184625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] RBAC: Adding query filter params (role.id=96417dd6dfef4a79b4cb31119e1cbc94, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d93e24780ff740b58285575d6efdffa6) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:19.184912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:19.185767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:19.186321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:19.186927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.187490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.187964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.188820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:19.189343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:19.190906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:19.192261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 442/908] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:19 2022] GET /identity/v3/role_assignments?user.id=d93e24780ff740b58285575d6efdffa6&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=96417dd6dfef4a79b4cb31119e1cbc94 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:19.196883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 467/909] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:19.205124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:19.317628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2c8d1-54e9-4e7d-bab7-1e7d253dbab0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:19.354527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:19.355169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.355448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.355714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.356339 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:19.356636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:19.357999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:19.363587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 443/910] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:19.369866 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:19.482892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52be23d-6e8a-4d81-a821-a680fa9db8cf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:19.507090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:19.519595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0194a47-88a6-4dda-816e-f1ab123ab3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:19.520206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194a47-88a6-4dda-816e-f1ab123ab3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.520486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194a47-88a6-4dda-816e-f1ab123ab3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.520759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0194a47-88a6-4dda-816e-f1ab123ab3c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.521533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0194a47-88a6-4dda-816e-f1ab123ab3c8 None admin] RBAC: Adding query filter params (interface=public, service_id=cd7c7065decd44a4a3e1ca5f26ad44f2, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:19.521842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0194a47-88a6-4dda-816e-f1ab123ab3c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:19.523240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0194a47-88a6-4dda-816e-f1ab123ab3c8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:19.529659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 468/911] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Aug 23 07:56:19 2022] GET /identity/v3/endpoints?service_id=cd7c7065decd44a4a3e1ca5f26ad44f2&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:19.536899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:19.549999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:19.644415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3be82-87cb-4e8e-a2e6-fc511e92c37a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:19.680285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c4302-45ad-4914-8f56-1942d46b11f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:19.681282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c4302-45ad-4914-8f56-1942d46b11f1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.681872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c4302-45ad-4914-8f56-1942d46b11f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.682357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c4302-45ad-4914-8f56-1942d46b11f1 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.683179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c4302-45ad-4914-8f56-1942d46b11f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:19.684740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c4302-45ad-4914-8f56-1942d46b11f1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:19.691357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 444/912] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:56:19 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:56:19.815390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d33e8-fcb9-48f7-8c50-42447eebf305 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.815719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d33e8-fcb9-48f7-8c50-42447eebf305 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.815948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d33e8-fcb9-48f7-8c50-42447eebf305 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.817666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 469/913] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:19.824714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79ac52-c941-4f9a-ae50-a94ef7896160 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:19.824714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79ac52-c941-4f9a-ae50-a94ef7896160 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:19.824714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79ac52-c941-4f9a-ae50-a94ef7896160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:19.843648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6f79ac52-c941-4f9a-ae50-a94ef7896160 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:19.844864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6f79ac52-c941-4f9a-ae50-a94ef7896160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:19.918464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 445/914] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:20.184882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d33e8-fcb9-48f7-8c50-42447eebf305 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:20.294013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d33e8-fcb9-48f7-8c50-42447eebf305 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:20.329474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:20.330104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:20.330436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:20.330715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:20.331324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:20.332696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:20.342093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:20.343184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 07:56:20.344023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:20.348611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 470/915] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:20.350104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79ac52-c941-4f9a-ae50-a94ef7896160 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:20.359183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:20.458077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79ac52-c941-4f9a-ae50-a94ef7896160 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:20.494059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4edadc-6f97-46ef-a800-bcc989de7098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:20.494780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4edadc-6f97-46ef-a800-bcc989de7098 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:20.495094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4edadc-6f97-46ef-a800-bcc989de7098 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:20.495374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4edadc-6f97-46ef-a800-bcc989de7098 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:20.496056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4edadc-6f97-46ef-a800-bcc989de7098 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:20.496363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4edadc-6f97-46ef-a800-bcc989de7098 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:20.497739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4edadc-6f97-46ef-a800-bcc989de7098 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:20.502931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 446/916] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:20.508589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:20.563778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:20.569642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8df4-179e-412d-a47f-85cf07510790 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:20.569642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8df4-179e-412d-a47f-85cf07510790 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:20.569642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8df4-179e-412d-a47f-85cf07510790 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:20.570981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 447/917] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:20.577669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712898f-dd53-48d3-9479-d0c73884f47f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:20.577945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712898f-dd53-48d3-9479-d0c73884f47f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:20.578212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712898f-dd53-48d3-9479-d0c73884f47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:20.602058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d712898f-dd53-48d3-9479-d0c73884f47f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:20.603434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d712898f-dd53-48d3-9479-d0c73884f47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:20.628218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb506d5-a8bc-4435-becd-a99e1a2dffc2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:20.665313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:20.666367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:20.666865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:20.667370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:20.668317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:20.668837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:20.670507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:20.681067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 471/918] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Aug 23 07:56:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:20.682992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:20.684816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:20.687246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 448/919] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:20.790493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5556a-26a2-4018-973b-08c8ac444e5f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:20.805509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:20.826688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5c698-0d06-47ab-bd8d-c8976703b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:20.827328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5c698-0d06-47ab-bd8d-c8976703b3c1 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:20.827606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5c698-0d06-47ab-bd8d-c8976703b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:20.827877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5c698-0d06-47ab-bd8d-c8976703b3c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:20.828535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5c698-0d06-47ab-bd8d-c8976703b3c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:20.829901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5c698-0d06-47ab-bd8d-c8976703b3c1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:20.939157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 472/920] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:56:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:56:20.980243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712898f-dd53-48d3-9479-d0c73884f47f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:21.089841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712898f-dd53-48d3-9479-d0c73884f47f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:21.133336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8db52-24f5-4b3f-ac32-21fa7b3f154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:21.134197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8db52-24f5-4b3f-ac32-21fa7b3f154e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.134622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8db52-24f5-4b3f-ac32-21fa7b3f154e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.135604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8db52-24f5-4b3f-ac32-21fa7b3f154e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.136350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8db52-24f5-4b3f-ac32-21fa7b3f154e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:21.137910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8db52-24f5-4b3f-ac32-21fa7b3f154e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-17a8db52-24f5-4b3f-ac32-21fa7b3f154e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:21.147120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:21.148329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:21.149371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:21.151798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 449/921] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:21.157766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5c698-0d06-47ab-bd8d-c8976703b3c1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:21.208087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468426e-54dc-43a2-84bf-68518e015e25 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.208660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468426e-54dc-43a2-84bf-68518e015e25 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.209176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468426e-54dc-43a2-84bf-68518e015e25 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.211129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 450/922] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:21.215988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c97e-80d7-40ac-ad56-a0e3de262a56 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.216501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c97e-80d7-40ac-ad56-a0e3de262a56 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.217092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c97e-80d7-40ac-ad56-a0e3de262a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.236815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-af03c97e-80d7-40ac-ad56-a0e3de262a56 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:21.238254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-af03c97e-80d7-40ac-ad56-a0e3de262a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:21.270867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5c698-0d06-47ab-bd8d-c8976703b3c1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:21.297996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:21.309554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:21.310391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.310753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.311233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.311969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:21.312372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:21.313835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:21.323118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 473/924] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:21.323373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 451/924] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:21.325273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:21.361478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:21.433899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc2ae6-4947-47f1-80a0-04c6039949ab None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:21.469757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:21.470657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.471111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.471544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.472312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:21.472779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:21.474290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:21.479965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 474/925] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:21.488067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03c97e-80d7-40ac-ad56-a0e3de262a56 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:21.594311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03c97e-80d7-40ac-ad56-a0e3de262a56 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:21.629851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:21.630417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.630700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.630996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.631756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] RBAC: Adding query filter params (interface=public, service_id=52054d5bf08e471d8daf2da185e38969, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:21.632060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:21.633478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:21.641270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 452/926] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Aug 23 07:56:21 2022] GET /identity/v3/endpoints?service_id=52054d5bf08e471d8daf2da185e38969&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:21.646580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:21.647877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:21.713221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:21.764937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:21.769035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc340ef5-6ca6-4867-8904-cea59f51b693 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:21.785250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9484ea1-3106-46c1-83dd-8899a248ebf9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:21.809991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd2ae8-4205-475b-bcb8-f3993dc4b990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:21.810973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd2ae8-4205-475b-bcb8-f3993dc4b990 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.811473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd2ae8-4205-475b-bcb8-f3993dc4b990 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.811805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd2ae8-4205-475b-bcb8-f3993dc4b990 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.812432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd2ae8-4205-475b-bcb8-f3993dc4b990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:21.813789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd2ae8-4205-475b-bcb8-f3993dc4b990 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:21.819791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 475/927] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:56:21 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:56:21.820925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc800cc-a883-42a0-808e-9e96db5fb1a1 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.821204 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc800cc-a883-42a0-808e-9e96db5fb1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.821478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc800cc-a883-42a0-808e-9e96db5fb1a1 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.823646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 476/928] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:21.829169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55101c2-78c3-4cbd-8f8a-31bbfd4afb0a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.829460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55101c2-78c3-4cbd-8f8a-31bbfd4afb0a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.829723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55101c2-78c3-4cbd-8f8a-31bbfd4afb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.830151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ac31-6b62-4d70-9efe-c1b7092141bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:21.831080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ac31-6b62-4d70-9efe-c1b7092141bc None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:21.831589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ac31-6b62-4d70-9efe-c1b7092141bc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:21.832060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9ac31-6b62-4d70-9efe-c1b7092141bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:21.846836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9ac31-6b62-4d70-9efe-c1b7092141bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:21.851238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b55101c2-78c3-4cbd-8f8a-31bbfd4afb0a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-4cc9ac31-6b62-4d70-9efe-c1b7092141bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:21.852399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:21.853345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:21.854753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:21.855441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b55101c2-78c3-4cbd-8f8a-31bbfd4afb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:21.862466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 453/929] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Aug 23 07:56:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:21.863782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ac31-6b62-4d70-9efe-c1b7092141bc None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:21.951087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 477/930] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:21.990312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9ac31-6b62-4d70-9efe-c1b7092141bc None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.004933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:22.026810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.027879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.028361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.028792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.029625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:22.030086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.031738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.040605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 454/931] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Aug 23 07:56:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:22.046907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55101c2-78c3-4cbd-8f8a-31bbfd4afb0a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.143341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:22.156860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55101c2-78c3-4cbd-8f8a-31bbfd4afb0a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.192373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.193005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.193364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.193638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.204970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.206375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:22.222302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:22.223270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:22.224147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:22.224147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:22.224147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:22.224147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 478/932] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:22.232293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.256241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.337174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d55497-54bf-4c50-95ff-512a2853e687 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.364346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c56778-647c-4773-98d9-13ce50874510 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.373135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.373745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.374036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.374630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.374950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:22.375279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.376660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.394189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 455/933] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:22.395590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.409429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.410381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.410886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.411393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.412221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.413844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:22.423709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:22.424629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 07:56:22.425504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:22.428195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 479/934] 10.222.0.58 () {56 vars in 1132 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:22.434231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:22.436570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.449705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:22.521282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf89e1c-594d-4294-9862-ac531a4a841d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.545408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ef7d8-34b1-4d97-b255-eaad265b55b7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.565108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.566017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.566301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.566560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.576964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.578811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.584790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.585758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.586238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.586681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.587532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:22.588142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.589723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:22.591609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:22.592563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:22.593521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:22.595089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 456/935] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:22.599587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.607426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 480/936] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:22.608325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.701512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:22.703302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:22.708639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd280b56-6ca7-4484-a467-515475644d2a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.725133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169dda5-7905-49f4-a2c7-26d05285d4ed None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.743994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.747888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.747888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.747888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.747888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:22.747888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.748799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.772879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 457/937] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:22.774550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.775621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.776293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.776569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.776887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.777527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:22.777828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.781800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.789909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 481/938] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:22.790780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.889242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687756-41b5-4345-b8d7-23e7861007ca None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.904012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d212e-2ab9-470c-8cf5-d1dd408af8dd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:22.929130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c7ff8-6569-44cf-94a4-72ded772e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.930208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7ff8-6569-44cf-94a4-72ded772e228 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.930733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7ff8-6569-44cf-94a4-72ded772e228 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.931269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7ff8-6569-44cf-94a4-72ded772e228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.941919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e234de0-2632-4adf-8505-9f29a8d15edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:22.942527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e234de0-2632-4adf-8505-9f29a8d15edc None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.942882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e234de0-2632-4adf-8505-9f29a8d15edc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.943128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e234de0-2632-4adf-8505-9f29a8d15edc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.943789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e234de0-2632-4adf-8505-9f29a8d15edc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.944249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c7ff8-6569-44cf-94a4-72ded772e228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:22.945209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e234de0-2632-4adf-8505-9f29a8d15edc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.946252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c7ff8-6569-44cf-94a4-72ded772e228 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:22.951381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b69c7ff8-6569-44cf-94a4-72ded772e228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:22.961419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:22.971514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 458/939] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc5ff-1d28-41af-95b1-531ee9642206 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc5ff-1d28-41af-95b1-531ee9642206 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc5ff-1d28-41af-95b1-531ee9642206 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 459/940] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:22.972396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dc5ff-1d28-41af-95b1-531ee9642206 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:22.974705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:23.057295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 482/941] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:56:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:23.059050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd154a5-7ddf-48d8-9ec9-a64ac93b4bb4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.059391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd154a5-7ddf-48d8-9ec9-a64ac93b4bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.059671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd154a5-7ddf-48d8-9ec9-a64ac93b4bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.084336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3cd154a5-7ddf-48d8-9ec9-a64ac93b4bb4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:23.085667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3cd154a5-7ddf-48d8-9ec9-a64ac93b4bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:23.095665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dc5ff-1d28-41af-95b1-531ee9642206 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:23.141253 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:23.141993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.142343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.142696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.143512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:23.144004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:23.145488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:23.165631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 460/942] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:23.168103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:23.184226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 483/943] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:23.240177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:23.284479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5547e9-87ae-4138-a0a3-c28ff362afbd None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:23.321922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:23.323096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.323534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.323974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.331476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:23.333088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:23.344870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:23.345957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:23.347105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:23.347105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:23.347105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:23.347105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:23.347105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:23.347105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:23.347105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:23.347105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.349100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 461/944] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:23.355671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd154a5-7ddf-48d8-9ec9-a64ac93b4bb4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:23.458698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd154a5-7ddf-48d8-9ec9-a64ac93b4bb4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:23.471595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:23.491330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:23.503373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:23.503896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.504223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.506543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.506543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:23.506543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:23.506901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:23.523460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 484/945] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:23.525704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:23.617459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c954f0-2bbc-4f42-a84c-258c77a9f6a3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:23.643149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:23.661509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:23.662110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.662440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.662789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.663615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:23.665311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:23.669416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING keystone.server.flask.common [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 07:56:23.676037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95969-8c53-4515-9284-acbb970ae941 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:23.715149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a06b41-d6d2-45d0-86b0-939cfb12d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:23.715964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06b41-d6d2-45d0-86b0-939cfb12d67c None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.716345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06b41-d6d2-45d0-86b0-939cfb12d67c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.716722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a06b41-d6d2-45d0-86b0-939cfb12d67c None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d93e24780ff740b58285575d6efdffa6/roles/96417dd6dfef4a79b4cb31119e1cbc94` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.719068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a06b41-d6d2-45d0-86b0-939cfb12d67c None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=d93e24780ff740b58285575d6efdffa6, role_id=96417dd6dfef4a79b4cb31119e1cbc94)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:23.720605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a06b41-d6d2-45d0-86b0-939cfb12d67c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:23.725280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.sql.core [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:56:23.725280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 07:56:23.725280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [parameters: {'id': '915c90ad59df49f0bfe2de447cfa0b4d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 07:56:23.725280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81607) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 915c90ad59df49f0bfe2de447cfa0b4d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 915c90ad59df49f0bfe2de447cfa0b4d. Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:56:23.741211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 23 07:56:23.742370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application yield resource Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise self.value Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application session.commit() Aug 23 07:56:23.743765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.session.flush() Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application compat.raise_( Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 07:56:23.745659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 07:56:23.748015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application result.read() Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:56:23.749895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [parameters: {'id': '915c90ad59df49f0bfe2de447cfa0b4d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:23.752184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:23.754117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 915c90ad59df49f0bfe2de447cfa0b4d. Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 462/946] 10.222.0.58 () {60 vars in 1140 bytes} [Tue Aug 23 07:56:23 2022] POST /identity/v3/projects => generated 262 bytes in 277 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:23.756793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:23.807075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 485/947] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:56:23 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/d93e24780ff740b58285575d6efdffa6/roles/96417dd6dfef4a79b4cb31119e1cbc94 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:56:23.852145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3eabd-0385-43e0-b04b-9daf55858fbb None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.852412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3eabd-0385-43e0-b04b-9daf55858fbb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.852678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3eabd-0385-43e0-b04b-9daf55858fbb None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.854554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 486/948] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:23.860232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75e15-4dfd-4c0c-b56b-6d7be0fe0dd9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.860535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75e15-4dfd-4c0c-b56b-6d7be0fe0dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.860780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e75e15-4dfd-4c0c-b56b-6d7be0fe0dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.884666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-55e75e15-4dfd-4c0c-b56b-6d7be0fe0dd9 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:23.886578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-55e75e15-4dfd-4c0c-b56b-6d7be0fe0dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:23.888053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385f990-6aeb-4755-95e1-3d6e6ce98797 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:23.937481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:23.943665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:23.948479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:23.948479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:23.948479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:23.954857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:23.957965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:23.975142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:23.976208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:23.977190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 463/949] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:23.983858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:23.984989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 487/950] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:24.021434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:24.090656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e16f92-e552-4b73-b190-bf0013bc0690 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:24.127992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:24.128606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:24.128883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:24.129227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:24.129940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:24.130302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:24.131781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:24.154933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 464/951] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:24.258154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e75e15-4dfd-4c0c-b56b-6d7be0fe0dd9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:24.370467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e75e15-4dfd-4c0c-b56b-6d7be0fe0dd9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:24.406028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:24.406579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:24.406851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:24.407144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:24.407724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:24.409080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:24.417931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:24.419213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:24.420101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 488/952] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:24.427216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:24.524526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:24.530587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d92b0-59e0-4eea-8e7b-05338376fb99 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:24.565479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:24.566098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:24.566374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:24.566645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:24.567303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:24.567601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:24.568973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:24.575421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 465/953] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:24.581141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:24.617914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:24.687593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecdea5-6a2b-4b4e-a246-269f462b0f80 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:24.723447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2817114-38ee-48ea-b475-e2f0a6745175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:24.724058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2817114-38ee-48ea-b475-e2f0a6745175 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:24.724339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2817114-38ee-48ea-b475-e2f0a6745175 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:24.724610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2817114-38ee-48ea-b475-e2f0a6745175 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:24.725236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2817114-38ee-48ea-b475-e2f0a6745175 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:24.725529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2817114-38ee-48ea-b475-e2f0a6745175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:24.726911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2817114-38ee-48ea-b475-e2f0a6745175 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:24.732846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 489/954] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:24.744320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:24.860890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dfd15-4583-4094-95e0-c9306be20297 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:24.898349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff55ec-d1f6-470d-89f2-9908f2cc6d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:24.899365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff55ec-d1f6-470d-89f2-9908f2cc6d4d None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:24.899868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff55ec-d1f6-470d-89f2-9908f2cc6d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:24.900940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff55ec-d1f6-470d-89f2-9908f2cc6d4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:24.901866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff55ec-d1f6-470d-89f2-9908f2cc6d4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:24.903474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff55ec-d1f6-470d-89f2-9908f2cc6d4d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:25.008608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 466/955] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:56:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:25.072036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bcb5b-6cde-4487-8fec-ba0095837ab9 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:25.072462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bcb5b-6cde-4487-8fec-ba0095837ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:25.072988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bcb5b-6cde-4487-8fec-ba0095837ab9 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:25.074682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 490/956] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:25.081334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145174b-c6d2-4f9f-928c-283ef48d4c30 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:25.081624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145174b-c6d2-4f9f-928c-283ef48d4c30 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:25.081888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145174b-c6d2-4f9f-928c-283ef48d4c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:25.102066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9145174b-c6d2-4f9f-928c-283ef48d4c30 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:25.103899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9145174b-c6d2-4f9f-928c-283ef48d4c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:25.116640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:25.186089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 467/957] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:25.454471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bcb5b-6cde-4487-8fec-ba0095837ab9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:25.555804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:25.562930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bcb5b-6cde-4487-8fec-ba0095837ab9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:25.601615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:25.602270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:25.602514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:25.602760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:25.603514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:25.605076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:25.625872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:25.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:25.627936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 491/958] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:25.633458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145174b-c6d2-4f9f-928c-283ef48d4c30 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:25.737477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145174b-c6d2-4f9f-928c-283ef48d4c30 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:25.773001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a6ae4-ae25-4d5c-bf2c-92bfa824e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:25.773607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6ae4-ae25-4d5c-bf2c-92bfa824e26d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:25.773887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6ae4-ae25-4d5c-bf2c-92bfa824e26d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:25.774161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6ae4-ae25-4d5c-bf2c-92bfa824e26d None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:25.774794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a6ae4-ae25-4d5c-bf2c-92bfa824e26d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:25.775127 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a6ae4-ae25-4d5c-bf2c-92bfa824e26d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:25.777213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a6ae4-ae25-4d5c-bf2c-92bfa824e26d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:25.786857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 468/959] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:25.788204 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:25.799350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9ffb-b90a-4562-af2b-5004fea1c971 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:25.799655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9ffb-b90a-4562-af2b-5004fea1c971 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:25.799888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9ffb-b90a-4562-af2b-5004fea1c971 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:25.801238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 469/960] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:25.807059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8bab8-4a97-426f-9bc5-53b1b80e9ecf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:25.807350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8bab8-4a97-426f-9bc5-53b1b80e9ecf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:25.807617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8bab8-4a97-426f-9bc5-53b1b80e9ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:25.821930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:25.827603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7db8bab8-4a97-426f-9bc5-53b1b80e9ecf None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:25.829335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7db8bab8-4a97-426f-9bc5-53b1b80e9ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:25.898142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2590-b2e4-404c-b0b8-7e380467ab27 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:25.923061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 470/961] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:25.934521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f9920-b853-489f-9ea3-3c9c857e8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:25.935246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f9920-b853-489f-9ea3-3c9c857e8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:25.935585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f9920-b853-489f-9ea3-3c9c857e8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:25.935924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f9920-b853-489f-9ea3-3c9c857e8ee4 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:25.936610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f9920-b853-489f-9ea3-3c9c857e8ee4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:25.937023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f9920-b853-489f-9ea3-3c9c857e8ee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:25.938471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f9920-b853-489f-9ea3-3c9c857e8ee4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:25.943750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 492/962] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:26.200458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8bab8-4a97-426f-9bc5-53b1b80e9ecf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:26.305848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8bab8-4a97-426f-9bc5-53b1b80e9ecf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:26.334432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a105457-def7-40d5-b424-a77a5348d9a7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:26.335113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a105457-def7-40d5-b424-a77a5348d9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:26.335624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a105457-def7-40d5-b424-a77a5348d9a7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:26.337429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 493/963] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:26.342898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf70cfe-3f0e-4af5-9ea0-3651ce1565e1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:26.343213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf70cfe-3f0e-4af5-9ea0-3651ce1565e1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:26.343487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf70cfe-3f0e-4af5-9ea0-3651ce1565e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:26.344599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c637fda-68b6-439c-860c-7deb0b5f044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:26.345463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c637fda-68b6-439c-860c-7deb0b5f044b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:26.345945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c637fda-68b6-439c-860c-7deb0b5f044b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:26.346370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c637fda-68b6-439c-860c-7deb0b5f044b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:26.349654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:26.359431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c637fda-68b6-439c-860c-7deb0b5f044b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:26.374965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fdf70cfe-3f0e-4af5-9ea0-3651ce1565e1 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:26.374965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fdf70cfe-3f0e-4af5-9ea0-3651ce1565e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-2c637fda-68b6-439c-860c-7deb0b5f044b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:26.375174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:26.376079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 471/964] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Aug 23 07:56:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:26.376929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c637fda-68b6-439c-860c-7deb0b5f044b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:26.392249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:26.441307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 494/965] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:26.496232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c637fda-68b6-439c-860c-7deb0b5f044b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:26.536504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:26.537491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:26.537981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:26.538458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:26.539396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:26.539928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:26.542037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:26.550980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 472/966] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Aug 23 07:56:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:26.562991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf70cfe-3f0e-4af5-9ea0-3651ce1565e1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:26.659418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf70cfe-3f0e-4af5-9ea0-3651ce1565e1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:26.694280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:26.694906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:26.695292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:26.695567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:26.706193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:26.707791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:26.722702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:26.723709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:26.724614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:26.724614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:26.724614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:26.724614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 495/967] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:26.730002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:26.750333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:26.763219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:26.802376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:26.858604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1de92-ea5d-4775-b821-5158a5283b14 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:26.883945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d8f8-1b63-44ab-a005-ebf1380f1e77 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:26.902224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:26.903329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:26.903605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:26.903876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:26.904761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:26.905084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:26.906687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:26.923025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 473/968] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:26.927211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:26.935016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:26.935620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:26.935891 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:26.936154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:26.936787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:26.938307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:26.940976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING keystone.server.flask.common [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 07:56:26.982389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.sql.core [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:56:26.982389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 07:56:26.982389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [parameters: {'user_id': 'ae043343128f4757a8285f74fb26f941', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 07:56:26.982389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81606) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result.read() Aug 23 07:56:26.992251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:56:26.993151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application yield resource Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application next(self.gen) Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise self.value Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application session.commit() Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 07:56:26.994355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.session.flush() Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application compat.raise_( Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 07:56:26.995807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 07:56:26.996694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application conn.query(q) Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application result.read() Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 07:56:26.998252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ae043343128f4757a8285f74fb26f941', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:27.000463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 23 07:56:27.002375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 23 07:56:27.004602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 23 07:56:27.004602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 23 07:56:27.004602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 07:56:27.004602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 07:56:27.004602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 07:56:27.004602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:27.004602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 496/969] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Aug 23 07:56:26 2022] POST /identity/v3/users => generated 160 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:27.004602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.055482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026165-edf2-4819-9a25-19e318ac84c7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:27.074548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:27.078459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:27.093363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797ff86-84ab-4a4e-a756-7fe8fb57f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:27.093992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ff86-84ab-4a4e-a756-7fe8fb57f94f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.094265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ff86-84ab-4a4e-a756-7fe8fb57f94f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.094586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ff86-84ab-4a4e-a756-7fe8fb57f94f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.102257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797ff86-84ab-4a4e-a756-7fe8fb57f94f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:27.104491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797ff86-84ab-4a4e-a756-7fe8fb57f94f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:27.115318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b6fc0-71fc-41d5-bf03-f7812976cc17 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0797ff86-84ab-4a4e-a756-7fe8fb57f94f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:27.115848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:27.116842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 07:56:27.117839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:27.122040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 474/970] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:56:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:27.123151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8011feb-61a5-4427-84c1-ca92ee8a30a6 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.123434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8011feb-61a5-4427-84c1-ca92ee8a30a6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.123704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8011feb-61a5-4427-84c1-ca92ee8a30a6 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.125208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 475/971] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:27.125905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8011feb-61a5-4427-84c1-ca92ee8a30a6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.155388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171a9c2-4544-41ff-a685-d855f7495048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:27.156175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a9c2-4544-41ff-a685-d855f7495048 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.156553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a9c2-4544-41ff-a685-d855f7495048 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.156927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a9c2-4544-41ff-a685-d855f7495048 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.164641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171a9c2-4544-41ff-a685-d855f7495048 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:27.166968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171a9c2-4544-41ff-a685-d855f7495048 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2171a9c2-4544-41ff-a685-d855f7495048 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:27.178895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:27.180039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 07:56:27.181158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:27.186485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 497/972] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:56:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:27.187894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470987a-b3b3-4b3e-8837-bd42d1354027 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.188178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470987a-b3b3-4b3e-8837-bd42d1354027 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.188450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470987a-b3b3-4b3e-8837-bd42d1354027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.206656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0470987a-b3b3-4b3e-8837-bd42d1354027 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:27.208247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0470987a-b3b3-4b3e-8837-bd42d1354027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:27.237850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8011feb-61a5-4427-84c1-ca92ee8a30a6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:27.273378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:27.274231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.274671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.274958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.275776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:27.276074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:27.277479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:27.283138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 498/973] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:27.283138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470987a-b3b3-4b3e-8837-bd42d1354027 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.311529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 476/974] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Aug 23 07:56:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:27.317025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.357644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:27.364610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:27.397569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470987a-b3b3-4b3e-8837-bd42d1354027 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:27.436696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:27.437543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.438649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.439114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.440040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:27.440541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:27.442166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:27.448982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64996f0-4625-4fae-84c9-cd876af109ad None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:27.485477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 499/975] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Aug 23 07:56:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:27.493881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b687ce-7387-4e57-8b0c-997656787f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:27.493881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b687ce-7387-4e57-8b0c-997656787f08 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.493881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b687ce-7387-4e57-8b0c-997656787f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.493881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b687ce-7387-4e57-8b0c-997656787f08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.563267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.579769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b687ce-7387-4e57-8b0c-997656787f08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:27.581738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b687ce-7387-4e57-8b0c-997656787f08 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:27.632216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:27.683607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a2193-a7bb-48e8-9a1b-16fa7960d4b7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:27.709672 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 477/976] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:56:27.711655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:27.714481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b687ce-7387-4e57-8b0c-997656787f08 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.733063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:27.733695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.738022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.738022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.738022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:27.738022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:27.745860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:27.746989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:27.747837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 500/977] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:27.754326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.845997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b687ce-7387-4e57-8b0c-997656787f08 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:27.870984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e93edf-6cb1-4f74-ae35-9094d891412e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:27.882666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:27.883328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.883603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.883871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.886665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:27.894406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:27.895909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:27.910952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:27.911922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:27.912798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:27.912798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:27.912798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:27.912798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 478/978] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:27.917024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:27.917937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:27.918209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:27.918320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.918492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:27.919158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:27.919465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:27.920847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:27.930987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 501/979] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:27.932226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:27.982927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:28.020080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f5401-e79c-457a-80e9-c1a18b929a65 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:28.034171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329ed62-5c3b-465f-ac5c-58a1147148f4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:28.053457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56090ec6-5b95-41ca-818d-5a622629a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:28.054063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56090ec6-5b95-41ca-818d-5a622629a56e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.054340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56090ec6-5b95-41ca-818d-5a622629a56e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.054611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56090ec6-5b95-41ca-818d-5a622629a56e None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.055320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56090ec6-5b95-41ca-818d-5a622629a56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:28.055626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56090ec6-5b95-41ca-818d-5a622629a56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:28.057588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56090ec6-5b95-41ca-818d-5a622629a56e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:28.068225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:28.068823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.069102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.069374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.069993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:28.070298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:28.071680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:28.073867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 479/980] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:28.075197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0727c-4037-4469-a2c3-16d6ce32d437 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.075483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0727c-4037-4469-a2c3-16d6ce32d437 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.075751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0727c-4037-4469-a2c3-16d6ce32d437 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.082744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 502/981] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:28.082899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 480/982] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:28.083014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:28.083127 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0727c-4037-4469-a2c3-16d6ce32d437 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:28.135586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:28.178504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e39889-e65d-4809-8822-eaadd5c03815 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:28.186282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0727c-4037-4469-a2c3-16d6ce32d437 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:28.191784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:28.213918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:28.214533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.214811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.215111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.220800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f56c6-6c66-4454-88da-2fec28dc1444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:28.221397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f56c6-6c66-4454-88da-2fec28dc1444 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.221675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f56c6-6c66-4454-88da-2fec28dc1444 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.221945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f56c6-6c66-4454-88da-2fec28dc1444 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.222207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:28.222649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f56c6-6c66-4454-88da-2fec28dc1444 None admin] RBAC: Adding query filter params (interface=admin, service_id=52054d5bf08e471d8daf2da185e38969, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:28.222977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f56c6-6c66-4454-88da-2fec28dc1444 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:28.223686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:28.224342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f56c6-6c66-4454-88da-2fec28dc1444 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:28.229824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 481/983] 10.222.0.58 () {56 vars in 1249 bytes} [Tue Aug 23 07:56:28 2022] GET /identity/v3/endpoints?service_id=52054d5bf08e471d8daf2da185e38969&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:28.230976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbdbb-3f3f-4228-bfc1-0e8d2ec4dc8c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.231262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbdbb-3f3f-4228-bfc1-0e8d2ec4dc8c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.231537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fbdbb-3f3f-4228-bfc1-0e8d2ec4dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:28.235488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:28.236541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 503/984] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:28.237452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:28.252611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-222fbdbb-3f3f-4228-bfc1-0e8d2ec4dc8c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:28.253792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-222fbdbb-3f3f-4228-bfc1-0e8d2ec4dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:28.320375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 482/985] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:28.321327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fbdbb-3f3f-4228-bfc1-0e8d2ec4dc8c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:28.335451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274394a4-acb1-4727-8408-75498f127d61 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:28.368659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792cf75-f305-48e6-9ea2-97e84940002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:28.369802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792cf75-f305-48e6-9ea2-97e84940002b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.370076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792cf75-f305-48e6-9ea2-97e84940002b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.370342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792cf75-f305-48e6-9ea2-97e84940002b None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.370933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792cf75-f305-48e6-9ea2-97e84940002b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:28.372314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792cf75-f305-48e6-9ea2-97e84940002b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:28.378055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 504/986] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:56:28 2022] GET /identity/v3/services => generated 1497 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:56:28.380151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:28.444282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fbdbb-3f3f-4228-bfc1-0e8d2ec4dc8c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:28.481898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:28.482525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.482806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.483115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.483819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:28.484125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:28.485581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:28.503222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 483/987] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:28.504456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792cf75-f305-48e6-9ea2-97e84940002b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:28.545048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:28.588470 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:28.613234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792cf75-f305-48e6-9ea2-97e84940002b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:28.650149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335aab7c-7c7f-4036-89b1-6772cc15840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:28.650783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335aab7c-7c7f-4036-89b1-6772cc15840b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.651097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335aab7c-7c7f-4036-89b1-6772cc15840b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.651367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335aab7c-7c7f-4036-89b1-6772cc15840b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.652090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335aab7c-7c7f-4036-89b1-6772cc15840b None admin] RBAC: Adding query filter params (role.id=96417dd6dfef4a79b4cb31119e1cbc94, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=d93e24780ff740b58285575d6efdffa6) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:28.652400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335aab7c-7c7f-4036-89b1-6772cc15840b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:28.653820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335aab7c-7c7f-4036-89b1-6772cc15840b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:28.659835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 505/988] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:28 2022] GET /identity/v3/role_assignments?user.id=d93e24780ff740b58285575d6efdffa6&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=96417dd6dfef4a79b4cb31119e1cbc94 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:28.699499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790f88b-f989-4b9a-9dfb-93df9bb21a0d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:28.701336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:28.737299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d462484-070b-4684-b83f-9b80fe2e5c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:28.737924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d462484-070b-4684-b83f-9b80fe2e5c20 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:28.738205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d462484-070b-4684-b83f-9b80fe2e5c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:28.738496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d462484-070b-4684-b83f-9b80fe2e5c20 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:28.739189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d462484-070b-4684-b83f-9b80fe2e5c20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:28.740603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d462484-070b-4684-b83f-9b80fe2e5c20 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:28.823256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 484/989] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:56:28 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:56:30.299218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed18f39-baa9-41ea-b736-55e9b04b359e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:30.299742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed18f39-baa9-41ea-b736-55e9b04b359e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:30.299742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed18f39-baa9-41ea-b736-55e9b04b359e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:30.301149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 506/990] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:30.306480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e4cdc-855e-41c9-859d-97a798eabf25 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:30.306851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e4cdc-855e-41c9-859d-97a798eabf25 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:30.307057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e4cdc-855e-41c9-859d-97a798eabf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:30.325281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-af6e4cdc-855e-41c9-859d-97a798eabf25 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:30.326465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-af6e4cdc-855e-41c9-859d-97a798eabf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:30.402063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 485/991] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:30.579946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31687fc-538f-4895-a3ab-41a6c3711317 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:30.580132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31687fc-538f-4895-a3ab-41a6c3711317 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:30.580405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31687fc-538f-4895-a3ab-41a6c3711317 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:30.581987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 507/992] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:30.587158 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d237c8b-a452-4779-84aa-cd2cd2bebc4c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:30.587453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d237c8b-a452-4779-84aa-cd2cd2bebc4c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:30.587750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d237c8b-a452-4779-84aa-cd2cd2bebc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:30.606117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0d237c8b-a452-4779-84aa-cd2cd2bebc4c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:30.607307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0d237c8b-a452-4779-84aa-cd2cd2bebc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:30.670572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31687fc-538f-4895-a3ab-41a6c3711317 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:30.674983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 486/993] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:30.741799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d045639-8eb7-441b-bb5a-29b49c275251 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:30.742088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d045639-8eb7-441b-bb5a-29b49c275251 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:30.742399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d045639-8eb7-441b-bb5a-29b49c275251 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:30.748059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 487/994] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:30.749366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356cef5-68a9-4025-b055-f39e1157e2e3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:30.750079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356cef5-68a9-4025-b055-f39e1157e2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:30.750362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356cef5-68a9-4025-b055-f39e1157e2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:30.769812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b356cef5-68a9-4025-b055-f39e1157e2e3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:30.771069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b356cef5-68a9-4025-b055-f39e1157e2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:30.775686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:30.780615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31687fc-538f-4895-a3ab-41a6c3711317 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:30.787944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:30.820597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:30.821225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:30.821505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:30.821848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:30.823166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:30.824594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:30.833534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:30.834512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:30.835488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 508/995] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Aug 23 07:56:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:30.840934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:30.849647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 488/996] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:30.939181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356cef5-68a9-4025-b055-f39e1157e2e3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:30.941362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d9737-600f-44d8-a753-fe1dd45006e6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:30.977142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:30.977745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:30.978025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:30.978325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:30.978975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:30.979274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:30.980636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:30.985330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 509/997] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:30.991364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:31.020605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:31.050545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356cef5-68a9-4025-b055-f39e1157e2e3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.087033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.088125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.088412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.088681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] PATH_INFO: `/v3/services/image` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.089268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.090644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.099390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d925fb-97a2-4aca-b67b-02ee61d9a0cf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:31.099535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:31.100504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:31.101352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 489/998] 10.222.0.58 () {56 vars in 1105 bytes} [Tue Aug 23 07:56:30 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:31.106448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:31.122306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:31.137822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.138424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.138706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.139005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.139629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:31.139934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.141350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.146087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 510/999] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Aug 23 07:56:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:31.146877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:31.210210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15436-b4c3-4ce3-95d3-b1852909cfdc None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.236367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:31.243167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.244290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.244617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.244884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.245026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d9683-92a9-4fb6-a011-0d23aeb49080 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.245503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] RBAC: Adding query filter params (name=image) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:31.245800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.247186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.251857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 490/1000] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:31.253403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:31.278192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.279153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.279433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.279713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.280501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.282171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:31.291349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:31.292531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:31.293525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 511/1001] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:31.295844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:31.332344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:31.361807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a52937-c742-44bf-b167-fb6b64fcb8ca None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.387994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba3cb-5d73-445b-bda0-4006f6503ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.388595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba3cb-5d73-445b-bda0-4006f6503ab0 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.388874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba3cb-5d73-445b-bda0-4006f6503ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.389146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba3cb-5d73-445b-bda0-4006f6503ab0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.389793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbba3cb-5d73-445b-bda0-4006f6503ab0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.391185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbba3cb-5d73-445b-bda0-4006f6503ab0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.395517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da5027-c14f-4f34-ab1e-4c1d7745be35 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.429792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.430387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.430678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.430965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.431614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] RBAC: Adding query filter params (type=image) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:31.431915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.433261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.437608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 512/1002] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:31.438371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:31.464384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:31.482978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 491/1003] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:56:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:31.572464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126a9de-3cd3-4579-99d8-16f851e25876 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.622173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.622700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.623011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.623269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.623887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:31.624185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.625578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.630492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 513/1004] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:31.636998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba3cb-5d73-445b-bda0-4006f6503ab0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:31.746200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba3cb-5d73-445b-bda0-4006f6503ab0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.787382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.787999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.788280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.788548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.789184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:31.789488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.791422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.793918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:31.801969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 492/1005] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:31.803739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:31.888737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:31.911063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a1b46-12ae-4125-a112-3180ac3e18a5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:31.945367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0716c-c494-43c6-b389-87d42478b5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:31.945966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0716c-c494-43c6-b389-87d42478b5dc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:31.946242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0716c-c494-43c6-b389-87d42478b5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:31.946502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0716c-c494-43c6-b389-87d42478b5dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:31.947246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0716c-c494-43c6-b389-87d42478b5dc None admin] RBAC: Adding query filter params (interface=public, service_id=c01b8f8ca7824d899a320de8e7e170d3, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:31.947547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0716c-c494-43c6-b389-87d42478b5dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:31.949170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0716c-c494-43c6-b389-87d42478b5dc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:31.954408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 514/1006] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Aug 23 07:56:31 2022] GET /identity/v3/endpoints?service_id=c01b8f8ca7824d899a320de8e7e170d3&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:31.959989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:32.078615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d734bb-778a-46df-9115-50b2ee333d5c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:32.111816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec0a89-1eee-4736-8f79-1cdda90436fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:32.112412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec0a89-1eee-4736-8f79-1cdda90436fe None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:32.112687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec0a89-1eee-4736-8f79-1cdda90436fe None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:32.112956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec0a89-1eee-4736-8f79-1cdda90436fe None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:32.113549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec0a89-1eee-4736-8f79-1cdda90436fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:32.114904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec0a89-1eee-4736-8f79-1cdda90436fe None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:32.121701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 493/1007] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:56:31 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:56:33.342259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad86f68-3373-4717-9f2a-b762728fda05 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:33.342809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad86f68-3373-4717-9f2a-b762728fda05 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:33.342809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad86f68-3373-4717-9f2a-b762728fda05 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:33.344199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 515/1008] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:33.349265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca676e-85bd-44bf-a19b-54febf4dbfe3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:33.349662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca676e-85bd-44bf-a19b-54febf4dbfe3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:33.349843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca676e-85bd-44bf-a19b-54febf4dbfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:33.368357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2eca676e-85bd-44bf-a19b-54febf4dbfe3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:33.370048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2eca676e-85bd-44bf-a19b-54febf4dbfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:33.429201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:33.438521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 494/1009] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:33.451649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91e673-6a1b-4cae-a554-e80c56f16246 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:33.451926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91e673-6a1b-4cae-a554-e80c56f16246 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:33.452200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91e673-6a1b-4cae-a554-e80c56f16246 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:33.453698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 516/1010] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:33.459079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076ebd-38e6-4248-b8ca-b4c5e4070037 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:33.459399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076ebd-38e6-4248-b8ca-b4c5e4070037 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:33.459891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076ebd-38e6-4248-b8ca-b4c5e4070037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:33.477603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c4076ebd-38e6-4248-b8ca-b4c5e4070037 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:33.478815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c4076ebd-38e6-4248-b8ca-b4c5e4070037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:33.545445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 495/1011] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:33.709173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91e673-6a1b-4cae-a554-e80c56f16246 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:33.814541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4076ebd-38e6-4248-b8ca-b4c5e4070037 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:33.822088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91e673-6a1b-4cae-a554-e80c56f16246 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:33.857761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1196b-2ae9-4a6a-89a4-eaec5598722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:33.858399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1196b-2ae9-4a6a-89a4-eaec5598722a None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:33.858667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1196b-2ae9-4a6a-89a4-eaec5598722a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:33.858986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1196b-2ae9-4a6a-89a4-eaec5598722a None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:33.859637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1196b-2ae9-4a6a-89a4-eaec5598722a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:33.860991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1196b-2ae9-4a6a-89a4-eaec5598722a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:33.917193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4076ebd-38e6-4248-b8ca-b4c5e4070037 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:33.940137 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:33.951396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff587f-dd7e-477f-ba91-647f3dacd3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:33.952085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff587f-dd7e-477f-ba91-647f3dacd3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:33.952441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff587f-dd7e-477f-ba91-647f3dacd3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:33.952623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff587f-dd7e-477f-ba91-647f3dacd3d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:33.953796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff587f-dd7e-477f-ba91-647f3dacd3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:33.955227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff587f-dd7e-477f-ba91-647f3dacd3d0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:33.956673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 517/1012] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:56:33 2022] POST /identity/v3/services => generated 242 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:33.962110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 496/1013] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:33.964216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1196b-2ae9-4a6a-89a4-eaec5598722a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:33.968325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:34.001618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c15cd-87c8-44c4-a38d-4bab44f32f08 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:34.001905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c15cd-87c8-44c4-a38d-4bab44f32f08 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:34.002179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c15cd-87c8-44c4-a38d-4bab44f32f08 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:34.003630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 497/1014] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:34.010007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0583-7031-4cfc-bc7d-3a272dd363d0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:34.010324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0583-7031-4cfc-bc7d-3a272dd363d0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:34.010552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0583-7031-4cfc-bc7d-3a272dd363d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:34.029685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e65b0583-7031-4cfc-bc7d-3a272dd363d0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:34.030898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e65b0583-7031-4cfc-bc7d-3a272dd363d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:34.093802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1196b-2ae9-4a6a-89a4-eaec5598722a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:34.132583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 498/1015] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:34.137607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f857a-251c-4924-b478-d25b55e075cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:34.138213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f857a-251c-4924-b478-d25b55e075cc None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:34.138488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f857a-251c-4924-b478-d25b55e075cc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:34.138808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f857a-251c-4924-b478-d25b55e075cc None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:34.139484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f857a-251c-4924-b478-d25b55e075cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:34.140904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f857a-251c-4924-b478-d25b55e075cc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:34.306920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 518/1016] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:56:33 2022] POST /identity/v3/projects => generated 311 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:34.404937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0583-7031-4cfc-bc7d-3a272dd363d0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:34.509534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0583-7031-4cfc-bc7d-3a272dd363d0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:34.533261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:34.545315 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:34.546164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:34.553971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:34.553971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:34.553971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:34.553971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:34.557771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:34.558740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:34.560020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 499/1017] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:34.564894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f857a-251c-4924-b478-d25b55e075cc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:34.622733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:34.671582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f857a-251c-4924-b478-d25b55e075cc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:34.705883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:34.706492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:34.706808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:34.707141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:34.707768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:34.708102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:34.709467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:34.719393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 519/1018] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:34.720530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:34.829503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284cdc0-2e06-4d36-b5ff-7c7ec139ec62 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:34.863773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a62ae5-9302-4ee1-9e7c-b13088931ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:34.864412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a62ae5-9302-4ee1-9e7c-b13088931ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:34.864654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a62ae5-9302-4ee1-9e7c-b13088931ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:34.864928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a62ae5-9302-4ee1-9e7c-b13088931ec2 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:34.865551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a62ae5-9302-4ee1-9e7c-b13088931ec2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:34.865854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a62ae5-9302-4ee1-9e7c-b13088931ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:34.867288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a62ae5-9302-4ee1-9e7c-b13088931ec2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:34.872324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 500/1019] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Aug 23 07:56:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:34.877872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:34.979145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517c321-902a-40cb-b140-ab054f2f0268 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:34.996471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:35.014812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e1665-512b-455f-a0d8-156a044a6b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:35.015441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e1665-512b-455f-a0d8-156a044a6b57 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:35.015725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e1665-512b-455f-a0d8-156a044a6b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:35.016000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e1665-512b-455f-a0d8-156a044a6b57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:35.016747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e1665-512b-455f-a0d8-156a044a6b57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:35.018111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e1665-512b-455f-a0d8-156a044a6b57 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:35.094187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 520/1020] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:56:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:35.900490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bbced-45a2-4696-a764-07a410392f35 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:35.901000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bbced-45a2-4696-a764-07a410392f35 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:35.901109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bbced-45a2-4696-a764-07a410392f35 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:35.902525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 501/1021] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:35.907710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc209ff-3bb3-4115-9a94-812c22792da2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:35.908088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc209ff-3bb3-4115-9a94-812c22792da2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:35.908268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc209ff-3bb3-4115-9a94-812c22792da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:35.927301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-abc209ff-3bb3-4115-9a94-812c22792da2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:35.928481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-abc209ff-3bb3-4115-9a94-812c22792da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:36.003099 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 521/1022] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:36.233452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd52e6-27d5-4e70-8e14-8bb595074ee7 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:36.233800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd52e6-27d5-4e70-8e14-8bb595074ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:36.233988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd52e6-27d5-4e70-8e14-8bb595074ee7 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:36.235577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 502/1023] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:36.241198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac19f0-8d20-46f6-afa0-3795bca5ac6e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:36.241483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac19f0-8d20-46f6-afa0-3795bca5ac6e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:36.241752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac19f0-8d20-46f6-afa0-3795bca5ac6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:36.259836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3fac19f0-8d20-46f6-afa0-3795bca5ac6e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:36.260999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3fac19f0-8d20-46f6-afa0-3795bca5ac6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:36.274813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd52e6-27d5-4e70-8e14-8bb595074ee7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:36.327547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:36.331966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 522/1024] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:36.378424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd52e6-27d5-4e70-8e14-8bb595074ee7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:36.411755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:36.412381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:36.412664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:36.412937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] PATH_INFO: `/v3/services/image` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:36.413516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:36.415050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:36.423616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:36.424615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:36.428025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 503/1025] 10.222.0.58 () {56 vars in 1105 bytes} [Tue Aug 23 07:56:36 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:36.430901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac19f0-8d20-46f6-afa0-3795bca5ac6e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:36.460407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:36.531772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac19f0-8d20-46f6-afa0-3795bca5ac6e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:36.565144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:36.565743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:36.566013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:36.566276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:36.566897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] RBAC: Adding query filter params (name=image) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:36.567222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:36.568683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:36.573984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 523/1026] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:36.579635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:36.599663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:36.683471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a215ed-e86f-4f0b-bf1b-132fb0842d5a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:36.694302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:36.698153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:36.704635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82132df-4e6b-4e92-8b31-b46aee63af05 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:36.717880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:36.718482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:36.718758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:36.719054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:36.719726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] RBAC: Adding query filter params (type=image) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:36.720027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:36.721381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:36.731321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 504/1027] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:36.732693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:36.747547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda343f-6bbe-464a-aa63-cf14b56c8dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:36.748232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda343f-6bbe-464a-aa63-cf14b56c8dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:36.748541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda343f-6bbe-464a-aa63-cf14b56c8dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:36.748777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda343f-6bbe-464a-aa63-cf14b56c8dc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:36.749938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda343f-6bbe-464a-aa63-cf14b56c8dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:36.751403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda343f-6bbe-464a-aa63-cf14b56c8dc3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:36.757590 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 524/1028] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:36.760166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda343f-6bbe-464a-aa63-cf14b56c8dc3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:36.840749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef65f6-d2f6-4895-bc00-dfc1530b4d6b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:36.871436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda343f-6bbe-464a-aa63-cf14b56c8dc3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:36.876220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:36.876820 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:36.877091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:36.877355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:36.878038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] RBAC: Adding query filter params (interface=public, service_id=ac324b27f8a74fd8ae4e9638af725b53, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:36.878341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:36.879723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:36.884893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 505/1029] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Aug 23 07:56:36 2022] GET /identity/v3/endpoints?service_id=ac324b27f8a74fd8ae4e9638af725b53&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:36.890142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:36.906280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804ae7a-2111-4b28-96f4-5ca77b6899d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:36.906895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804ae7a-2111-4b28-96f4-5ca77b6899d5 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:36.907200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804ae7a-2111-4b28-96f4-5ca77b6899d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:36.907473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804ae7a-2111-4b28-96f4-5ca77b6899d5 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:36.908110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804ae7a-2111-4b28-96f4-5ca77b6899d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:36.909540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804ae7a-2111-4b28-96f4-5ca77b6899d5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:36.935989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:36.991506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204b6eb-efec-40a8-a22e-6920744311a1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:37.024575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d5b7-b8f4-47e7-b710-31bd20efc111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:37.031166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 525/1030] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:56:36 2022] POST /identity/v3/users => generated 285 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:37.031334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d5b7-b8f4-47e7-b710-31bd20efc111 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:37.031334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d5b7-b8f4-47e7-b710-31bd20efc111 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:37.031334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d5b7-b8f4-47e7-b710-31bd20efc111 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:37.031334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68d5b7-b8f4-47e7-b710-31bd20efc111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:37.031334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68d5b7-b8f4-47e7-b710-31bd20efc111 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:37.035815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 506/1031] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:56:36 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 07:56:37.063744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7fe2f-b526-427c-b35d-a4ec5afb1887 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:37.064088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7fe2f-b526-427c-b35d-a4ec5afb1887 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:37.064437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7fe2f-b526-427c-b35d-a4ec5afb1887 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:37.066113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 526/1032] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:37.073017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea970-a1d8-480e-9359-fcf568bde7cb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:37.073346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea970-a1d8-480e-9359-fcf568bde7cb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:37.073627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea970-a1d8-480e-9359-fcf568bde7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:37.093636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9faea970-a1d8-480e-9359-fcf568bde7cb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:37.094911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9faea970-a1d8-480e-9359-fcf568bde7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:37.175926 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 507/1033] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:37.441986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7fe2f-b526-427c-b35d-a4ec5afb1887 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:37.464441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:37.548160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7fe2f-b526-427c-b35d-a4ec5afb1887 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:37.584609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6d7f2-c8a5-4e27-9915-30dcc3c3d349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:37.585245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6d7f2-c8a5-4e27-9915-30dcc3c3d349 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:37.585526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6d7f2-c8a5-4e27-9915-30dcc3c3d349 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:37.585798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6d7f2-c8a5-4e27-9915-30dcc3c3d349 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:37.596231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6d7f2-c8a5-4e27-9915-30dcc3c3d349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-77d6d7f2-c8a5-4e27-9915-30dcc3c3d349 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:37.601012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:37.601961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:37.603523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 527/1034] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Aug 23 07:56:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:37.608549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faea970-a1d8-480e-9359-fcf568bde7cb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:37.644428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:37.708013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faea970-a1d8-480e-9359-fcf568bde7cb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:37.741419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:37.742015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:37.742288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:37.742553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:37.743330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:37.743687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:37.745041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:37.758993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 508/1035] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Aug 23 07:56:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:37.759640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6d7f2-c8a5-4e27-9915-30dcc3c3d349 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:37.838111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:37.859616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6d7f2-c8a5-4e27-9915-30dcc3c3d349 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:37.893682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:37.894291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:37.894563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:37.894826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:37.904797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:37.906231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:37.920389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:37.921766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:37.922673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:37.922673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:37.922673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:37.923385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 528/1036] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:37.927303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:38.023459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:38.028681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd99cf-f141-4a81-bf26-a13ee66b8569 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:38.061621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:38.062282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:38.062665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:38.062875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:38.063534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:38.063830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:38.065188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:38.078692 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 509/1037] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:38.084092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:38.186146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0c80a-467c-4342-bf9c-4c4e1aa6625a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:38.213987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:38.222210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:38.233470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:38.233470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:38.233470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:38.233470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:38.233470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:38.243329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:38.244872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:38.245834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:38.249772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 529/1038] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:56:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:38.250828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:38.350898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb322f45-a928-4567-9aeb-9be37343607e None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:38.385040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:38.385649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:38.385928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:38.386202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:38.386923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:38.387269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:38.388682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:38.410217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 510/1039] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:56:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:38.411951 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:38.514777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd5c6a-7699-439d-b44b-d979c50b3f67 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:38.548818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8c50-30f4-4a3a-9f68-5b84d4639720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:38.549423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8c50-30f4-4a3a-9f68-5b84d4639720 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:38.549697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8c50-30f4-4a3a-9f68-5b84d4639720 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:38.549961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8c50-30f4-4a3a-9f68-5b84d4639720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:38.607387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:38.620058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e8c50-30f4-4a3a-9f68-5b84d4639720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:38.622395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e8c50-30f4-4a3a-9f68-5b84d4639720 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:38.723408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 530/1040] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:56:38.729482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:38.748702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:38.839087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21843a8e-863f-4aac-bbe2-359c5f7ff630 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:38.873372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8fe16-26ce-4083-b8e8-aad569ed0663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:38.873920 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fe16-26ce-4083-b8e8-aad569ed0663 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:38.874193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fe16-26ce-4083-b8e8-aad569ed0663 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:38.874457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8fe16-26ce-4083-b8e8-aad569ed0663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:38.885216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8fe16-26ce-4083-b8e8-aad569ed0663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:38.886652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8fe16-26ce-4083-b8e8-aad569ed0663 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-09b8fe16-26ce-4083-b8e8-aad569ed0663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:38.901133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:38.902117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:38.903053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:38.903053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:38.903053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:38.906914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 511/1041] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:38.908319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8c50-30f4-4a3a-9f68-5b84d4639720 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:38.978764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2430816-bffe-4054-aebe-b3c21ea51071 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:38.979126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2430816-bffe-4054-aebe-b3c21ea51071 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:38.979334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2430816-bffe-4054-aebe-b3c21ea51071 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:38.980822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 512/1042] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:38.985580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd980a-1315-42ea-a127-34eedc6e8b43 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:38.985862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd980a-1315-42ea-a127-34eedc6e8b43 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:38.986136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd980a-1315-42ea-a127-34eedc6e8b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:38.999111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:39.005451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e0bd980a-1315-42ea-a127-34eedc6e8b43 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:39.006639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e0bd980a-1315-42ea-a127-34eedc6e8b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:39.014591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8c50-30f4-4a3a-9f68-5b84d4639720 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.048671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a295e3d-1373-40ff-b83a-fc4f477b2e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.049270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a295e3d-1373-40ff-b83a-fc4f477b2e07 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.049553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a295e3d-1373-40ff-b83a-fc4f477b2e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.049814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a295e3d-1373-40ff-b83a-fc4f477b2e07 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.050434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a295e3d-1373-40ff-b83a-fc4f477b2e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:39.050730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a295e3d-1373-40ff-b83a-fc4f477b2e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:39.052122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a295e3d-1373-40ff-b83a-fc4f477b2e07 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:39.063766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 531/1043] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:39.064977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19e20-2749-4fee-bf24-1399dcb5d558 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.065259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19e20-2749-4fee-bf24-1399dcb5d558 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.065529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19e20-2749-4fee-bf24-1399dcb5d558 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.066708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 532/1044] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:39.069733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b19e20-2749-4fee-bf24-1399dcb5d558 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.070912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:39.077134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 513/1045] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:39.078355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe905-673c-4709-975b-6ab06b84a4c1 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.078664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe905-673c-4709-975b-6ab06b84a4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.078948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe905-673c-4709-975b-6ab06b84a4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.096340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c02fe905-673c-4709-975b-6ab06b84a4c1 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:39.097510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c02fe905-673c-4709-975b-6ab06b84a4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:39.169296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 514/1046] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:39.181418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b19e20-2749-4fee-bf24-1399dcb5d558 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.216005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.216628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.216904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.217174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.223732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:39.225177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:39.236674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:39.237530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:39.238435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 533/1047] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:39.243549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fe905-673c-4709-975b-6ab06b84a4c1 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.343806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.343997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fe905-673c-4709-975b-6ab06b84a4c1 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.354920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:39.372024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:39.380174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80037637-5039-46f2-af12-41520f63de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.380779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037637-5039-46f2-af12-41520f63de61 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.381059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037637-5039-46f2-af12-41520f63de61 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.381333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037637-5039-46f2-af12-41520f63de61 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.382020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80037637-5039-46f2-af12-41520f63de61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:39.382324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80037637-5039-46f2-af12-41520f63de61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:39.383786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80037637-5039-46f2-af12-41520f63de61 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:39.401228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 515/1048] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:39.402250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80037637-5039-46f2-af12-41520f63de61 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.451637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b9331-c2d3-48f7-bf2f-f51c565abd08 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.484042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.484661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.484931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.485197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.485765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:39.487136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:39.495992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:39.497191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:39.498043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 534/1049] 10.222.0.58 () {56 vars in 1105 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:39.498834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.506073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80037637-5039-46f2-af12-41520f63de61 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.539661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.540277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.540540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.540813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.541529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] RBAC: Adding query filter params (role.id=96417dd6dfef4a79b4cb31119e1cbc94, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=03cc4435b9b0444b9f278e15398443c7) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:39.541830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:39.543286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:39.555462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 516/1050] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/role_assignments?user.id=03cc4435b9b0444b9f278e15398443c7&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=96417dd6dfef4a79b4cb31119e1cbc94 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:39.555462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.570782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:39.606397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d340d-756b-40ac-853b-675d2bc715e7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.621071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:39.648624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30adf83-df56-4355-8a24-cc1356fcdcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.649366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30adf83-df56-4355-8a24-cc1356fcdcfe None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.649648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30adf83-df56-4355-8a24-cc1356fcdcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.650959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30adf83-df56-4355-8a24-cc1356fcdcfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.663240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30adf83-df56-4355-8a24-cc1356fcdcfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e30adf83-df56-4355-8a24-cc1356fcdcfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:39.668135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:39.669123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:39.670199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 535/1051] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:39.675658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30adf83-df56-4355-8a24-cc1356fcdcfe None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.682444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea686-66df-4945-aeb6-6f4bd1ce061a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.727209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.727652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.728471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.728744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.729372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] RBAC: Adding query filter params (name=image) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:39.729679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:39.731087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:39.736266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 517/1052] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:39.741970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.798260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30adf83-df56-4355-8a24-cc1356fcdcfe None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.830423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:39.835684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.836297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.836567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.836842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.837585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:39.837887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:39.839434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:39.847744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 536/1053] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:39.853954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.855364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2465d-b09b-45ab-aa9c-7b6fea75c422 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:39.873183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:39.892994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:39.893599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:39.893881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:39.894155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:39.894779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] RBAC: Adding query filter params (type=image) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:39.895133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:39.896514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:39.901363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 518/1054] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:39.907128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:39.964365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf7d3-0ecb-4be4-b3dc-8f6f4d4b638f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:40.000356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ae4af-9708-4ff9-88c9-46571305b766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:40.000953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ae4af-9708-4ff9-88c9-46571305b766 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:40.001182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ae4af-9708-4ff9-88c9-46571305b766 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:40.001422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ae4af-9708-4ff9-88c9-46571305b766 None admin] PATH_INFO: `/v3/users/0a38dee5d2844618a8edf311dfac56e5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:40.016449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ae4af-9708-4ff9-88c9-46571305b766 None admin] RBAC: Authorizing `identity:get_user(user_id=0a38dee5d2844618a8edf311dfac56e5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:40.018010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f06fa2-acbc-4aa8-a0ca-e9b3128a3c68 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:40.018415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ae4af-9708-4ff9-88c9-46571305b766 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:40.020452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 537/1055] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:39 2022] GET /identity/v3/users/0a38dee5d2844618a8edf311dfac56e5 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:40.026217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ae4af-9708-4ff9-88c9-46571305b766 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:40.054630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49473991-6f44-4cf5-a718-9fdea8cb5579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:40.055270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473991-6f44-4cf5-a718-9fdea8cb5579 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:40.055556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473991-6f44-4cf5-a718-9fdea8cb5579 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:40.055828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473991-6f44-4cf5-a718-9fdea8cb5579 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:40.056540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49473991-6f44-4cf5-a718-9fdea8cb5579 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:40.057903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49473991-6f44-4cf5-a718-9fdea8cb5579 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:40.082140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:40.128410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ae4af-9708-4ff9-88c9-46571305b766 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:40.161760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fdc744-fbc4-4ecb-9a09-51a6f3728943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:40.162364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdc744-fbc4-4ecb-9a09-51a6f3728943 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:40.162654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdc744-fbc4-4ecb-9a09-51a6f3728943 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:40.162908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdc744-fbc4-4ecb-9a09-51a6f3728943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:40.178868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 519/1056] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:56:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:56:40.238955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fdc744-fbc4-4ecb-9a09-51a6f3728943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:40.240606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fdc744-fbc4-4ecb-9a09-51a6f3728943 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:40.297015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:40.350810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 538/1057] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:56:40.356981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49473991-6f44-4cf5-a718-9fdea8cb5579 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:40.398375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:40.475881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49473991-6f44-4cf5-a718-9fdea8cb5579 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:40.510728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e273c-9442-4ed9-b19a-b51e10fdbe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:40.511404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e273c-9442-4ed9-b19a-b51e10fdbe09 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:40.511685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e273c-9442-4ed9-b19a-b51e10fdbe09 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:40.511958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e273c-9442-4ed9-b19a-b51e10fdbe09 None admin] PATH_INFO: `/v3/projects/e460c54fc5204a408e8564962f09c869` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:40.513080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e273c-9442-4ed9-b19a-b51e10fdbe09 None admin] RBAC: Authorizing `identity:get_project(project_id=e460c54fc5204a408e8564962f09c869)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:40.514611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e273c-9442-4ed9-b19a-b51e10fdbe09 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:40.521265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 520/1058] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:56:40 2022] GET /identity/v3/projects/e460c54fc5204a408e8564962f09c869 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:40.522851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fdc744-fbc4-4ecb-9a09-51a6f3728943 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:40.627522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fdc744-fbc4-4ecb-9a09-51a6f3728943 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:40.663193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c01fd-fd69-4b7b-9262-fb7587053ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:40.663798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c01fd-fd69-4b7b-9262-fb7587053ecd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:40.664074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c01fd-fd69-4b7b-9262-fb7587053ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:40.664341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c01fd-fd69-4b7b-9262-fb7587053ecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:40.665060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c01fd-fd69-4b7b-9262-fb7587053ecd None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=e460c54fc5204a408e8564962f09c869, user.id=0a38dee5d2844618a8edf311dfac56e5) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:40.665900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c01fd-fd69-4b7b-9262-fb7587053ecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:40.667346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c01fd-fd69-4b7b-9262-fb7587053ecd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:40.679209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 539/1059] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:40 2022] GET /identity/v3/role_assignments?user.id=0a38dee5d2844618a8edf311dfac56e5&scope.project.id=e460c54fc5204a408e8564962f09c869&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:41.411741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342343-d53f-4371-9dc0-854529d2df7c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:41.412258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342343-d53f-4371-9dc0-854529d2df7c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:41.412258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342343-d53f-4371-9dc0-854529d2df7c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:41.413603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 521/1060] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:41.419526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a183-4651-4782-8e1e-52cadf999c2a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:41.419934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a183-4651-4782-8e1e-52cadf999c2a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:41.420108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9a183-4651-4782-8e1e-52cadf999c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:41.438784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-15e9a183-4651-4782-8e1e-52cadf999c2a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:41.440024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-15e9a183-4651-4782-8e1e-52cadf999c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:41.451489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:41.506699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 540/1061] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:41.774347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc342343-d53f-4371-9dc0-854529d2df7c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:41.872530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:41.877095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc342343-d53f-4371-9dc0-854529d2df7c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:41.910100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399305d8-78d0-41d8-848c-184729964727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:41.910709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399305d8-78d0-41d8-848c-184729964727 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:41.911005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399305d8-78d0-41d8-848c-184729964727 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:41.911272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399305d8-78d0-41d8-848c-184729964727 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:41.921748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399305d8-78d0-41d8-848c-184729964727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-399305d8-78d0-41d8-848c-184729964727 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:41.926285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:41.927453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:41.928764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 522/1062] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Aug 23 07:56:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:41.936191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9a183-4651-4782-8e1e-52cadf999c2a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.036467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9a183-4651-4782-8e1e-52cadf999c2a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:42.069750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497534c0-25fe-4686-956e-ce7256ab96d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:42.070353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497534c0-25fe-4686-956e-ce7256ab96d8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.070623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497534c0-25fe-4686-956e-ce7256ab96d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.070886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497534c0-25fe-4686-956e-ce7256ab96d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.072084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497534c0-25fe-4686-956e-ce7256ab96d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:42.072381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497534c0-25fe-4686-956e-ce7256ab96d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:42.074000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497534c0-25fe-4686-956e-ce7256ab96d8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:42.086610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 541/1063] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Aug 23 07:56:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:42.088522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399305d8-78d0-41d8-848c-184729964727 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.119509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec9a47-ccce-4d31-994f-7ca660205879 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.119803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec9a47-ccce-4d31-994f-7ca660205879 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.120050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec9a47-ccce-4d31-994f-7ca660205879 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.121390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 542/1064] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:56:42.126334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af44a3c-328f-48a4-b634-c35abb133d7b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.126618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af44a3c-328f-48a4-b634-c35abb133d7b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.126893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af44a3c-328f-48a4-b634-c35abb133d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.147604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3af44a3c-328f-48a4-b634-c35abb133d7b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:42.149814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3af44a3c-328f-48a4-b634-c35abb133d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:42.153160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:42.194465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399305d8-78d0-41d8-848c-184729964727 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:42.233619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 543/1065] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:56:42.235742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:42.236382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.236656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.236925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.248667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:42.250101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:42.264281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:42.265207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:42.271162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:42.271162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:42.271162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:42.271162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 523/1066] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:42.271413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af44a3c-328f-48a4-b634-c35abb133d7b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.371689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af44a3c-328f-48a4-b634-c35abb133d7b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:42.406063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:42.406667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.406955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.407233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.407868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:42.408167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:42.409526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:42.416661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:42.422164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 544/1067] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:42.427812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.443977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:42.501580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.532667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf70a83-798d-4d4c-bd52-47805bf48dee None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:42.567860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c35a5-5442-4f20-a81f-d2636820f9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:42.568496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c35a5-5442-4f20-a81f-d2636820f9ee None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.568774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c35a5-5442-4f20-a81f-d2636820f9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.569046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c35a5-5442-4f20-a81f-d2636820f9ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.577909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c35a5-5442-4f20-a81f-d2636820f9ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:42.579589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c35a5-5442-4f20-a81f-d2636820f9ee None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-443c35a5-5442-4f20-a81f-d2636820f9ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:42.590607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:42.591760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:42.592927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:42.594720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 524/1068] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:56:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:42.595882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8005f-3207-4ce2-a321-731f7625b1a0 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.596159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8005f-3207-4ce2-a321-731f7625b1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.596423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8005f-3207-4ce2-a321-731f7625b1a0 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.597704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 525/1069] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:42.601022 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8005f-3207-4ce2-a321-731f7625b1a0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.607290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a70c7a-6cd7-40ef-9759-92af9ddf08b2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:42.641327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c3c19-85e3-470f-b6f3-8c7b0fd996c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:42.641857 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c3c19-85e3-470f-b6f3-8c7b0fd996c8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.642131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c3c19-85e3-470f-b6f3-8c7b0fd996c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.642401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c3c19-85e3-470f-b6f3-8c7b0fd996c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.671343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:42.700945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8005f-3207-4ce2-a321-731f7625b1a0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:42.704835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:42.708225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c3c19-85e3-470f-b6f3-8c7b0fd996c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:42.709682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c3c19-85e3-470f-b6f3-8c7b0fd996c8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:42.734680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760db6d-87ca-4a6d-a06c-b0dd616e673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:42.735314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760db6d-87ca-4a6d-a06c-b0dd616e673e None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.735598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760db6d-87ca-4a6d-a06c-b0dd616e673e None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.735876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760db6d-87ca-4a6d-a06c-b0dd616e673e None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.736593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760db6d-87ca-4a6d-a06c-b0dd616e673e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:42.736919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760db6d-87ca-4a6d-a06c-b0dd616e673e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:42.738338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760db6d-87ca-4a6d-a06c-b0dd616e673e None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:42.756321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 526/1070] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:56:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:42.757545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08959006-3a0c-4543-88bd-5465240e3789 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.757829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08959006-3a0c-4543-88bd-5465240e3789 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.758103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08959006-3a0c-4543-88bd-5465240e3789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.775310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-08959006-3a0c-4543-88bd-5465240e3789 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:42.776532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-08959006-3a0c-4543-88bd-5465240e3789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:42.814316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 545/1071] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:56:42.815517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c3c19-85e3-470f-b6f3-8c7b0fd996c8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.841745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 527/1072] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:42.842584 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08959006-3a0c-4543-88bd-5465240e3789 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.906710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:42.912387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:42.916813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c3c19-85e3-470f-b6f3-8c7b0fd996c8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:42.952141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08959006-3a0c-4543-88bd-5465240e3789 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:42.954502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:42.955164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.955446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.955719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:42.965776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:42.967249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:42.981996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:42.982979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:42.983888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:42.983888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:42.983888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:42.988345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 546/1073] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:42.989190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:42.990684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:42.991350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:42.991795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:42.992076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.004485 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.005926 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:43.021050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:43.022053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:43.023071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:43.023071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:43.023071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:43.023071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 528/1074] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:43.028034 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.096359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e403fc-9e60-4b39-8fa3-a9af6b9a4dfe None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.127794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c0349-7b68-4f18-8688-087a8b508211 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.129361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.129952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.130225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.130490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.131138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:43.131433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.132780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.140871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:43.144635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 547/1075] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:43.145410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.157032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:43.162016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.162626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.162906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.163208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.163843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:43.164146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.165539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.177488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 529/1076] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:43.179908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.258501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124515ba-1def-4797-a2f3-f3ce26ad5291 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.299186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b2eb5-bae8-4a4d-b664-17fe0a864ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.299551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ed6ea-398a-41a6-b875-076d649ed8fd None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.299823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b2eb5-bae8-4a4d-b664-17fe0a864ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.300095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b2eb5-bae8-4a4d-b664-17fe0a864ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.300361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b2eb5-bae8-4a4d-b664-17fe0a864ba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.313069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b2eb5-bae8-4a4d-b664-17fe0a864ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-514b2eb5-bae8-4a4d-b664-17fe0a864ba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:43.317721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:43.318648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:43.319537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 548/1077] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:43.325129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b2eb5-bae8-4a4d-b664-17fe0a864ba9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.346515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.347168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.347442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.347708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.357620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.359206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:43.373445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:43.374568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:43.375522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 530/1078] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:43.382752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.408860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:43.421375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:43.446326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b2eb5-bae8-4a4d-b664-17fe0a864ba9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.485798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.486403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.486680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.486962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.487634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:43.487932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.489427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.498042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 549/1079] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:56:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:43.498205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5aeaa4-703a-43ce-b7d0-93ed235352f3 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.503736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.535614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.536211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.536482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.536748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.537425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:43.537724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.539176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.551218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 531/1080] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:43.557739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.604657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2ce19-0de1-4b9e-876d-6ece950a97e3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.633615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:43.639065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247bc21-7e67-4484-b3d8-fa0f2ba63d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.639689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247bc21-7e67-4484-b3d8-fa0f2ba63d16 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.643580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247bc21-7e67-4484-b3d8-fa0f2ba63d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.643580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247bc21-7e67-4484-b3d8-fa0f2ba63d16 None admin] PATH_INFO: `/v3/users/0a38dee5d2844618a8edf311dfac56e5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.643580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247bc21-7e67-4484-b3d8-fa0f2ba63d16 None admin] RBAC: Authorizing `identity:get_user(user_id=0a38dee5d2844618a8edf311dfac56e5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.643580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247bc21-7e67-4484-b3d8-fa0f2ba63d16 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.644541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 550/1081] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:43 2022] GET /identity/v3/users/0a38dee5d2844618a8edf311dfac56e5 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:43.646486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:43.650144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247bc21-7e67-4484-b3d8-fa0f2ba63d16 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.655237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c77a2e-eada-4b59-b134-e52a2328ca74 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.689431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f092440-0d5c-4529-b534-50ca38a06ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.690086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092440-0d5c-4529-b534-50ca38a06ad0 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.690368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092440-0d5c-4529-b534-50ca38a06ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.690641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f092440-0d5c-4529-b534-50ca38a06ad0 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/03cc4435b9b0444b9f278e15398443c7/roles/96417dd6dfef4a79b4cb31119e1cbc94` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.692838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f092440-0d5c-4529-b534-50ca38a06ad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=03cc4435b9b0444b9f278e15398443c7, role_id=96417dd6dfef4a79b4cb31119e1cbc94)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.694268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f092440-0d5c-4529-b534-50ca38a06ad0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.747363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247bc21-7e67-4484-b3d8-fa0f2ba63d16 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.766878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 532/1082] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:56:43 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/03cc4435b9b0444b9f278e15398443c7/roles/96417dd6dfef4a79b4cb31119e1cbc94 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:56:43.782238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e267dc-5de7-473f-9c98-055d32b653ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.782729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e267dc-5de7-473f-9c98-055d32b653ed None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.783033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e267dc-5de7-473f-9c98-055d32b653ed None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.783306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e267dc-5de7-473f-9c98-055d32b653ed None admin] PATH_INFO: `/v3/projects/e460c54fc5204a408e8564962f09c869` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.784498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e267dc-5de7-473f-9c98-055d32b653ed None admin] RBAC: Authorizing `identity:get_project(project_id=e460c54fc5204a408e8564962f09c869)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.785979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e267dc-5de7-473f-9c98-055d32b653ed None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:43.787835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 551/1083] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:56:43 2022] GET /identity/v3/projects/e460c54fc5204a408e8564962f09c869 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:43.794157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f092440-0d5c-4529-b534-50ca38a06ad0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:43.898878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f092440-0d5c-4529-b534-50ca38a06ad0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:43.933011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75597f60-24f8-4ea5-bb43-a3a6605f4f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:43.933664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597f60-24f8-4ea5-bb43-a3a6605f4f03 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:43.933938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597f60-24f8-4ea5-bb43-a3a6605f4f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:43.934206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597f60-24f8-4ea5-bb43-a3a6605f4f03 None admin] PATH_INFO: `/v3/projects/e460c54fc5204a408e8564962f09c869/users/0a38dee5d2844618a8edf311dfac56e5/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:43.936110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75597f60-24f8-4ea5-bb43-a3a6605f4f03 None admin] RBAC: Authorizing `identity:create_grant(project_id=e460c54fc5204a408e8564962f09c869, user_id=0a38dee5d2844618a8edf311dfac56e5, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:43.937527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75597f60-24f8-4ea5-bb43-a3a6605f4f03 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:44.016398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 533/1084] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:56:43 2022] PUT /identity/v3/projects/e460c54fc5204a408e8564962f09c869/users/0a38dee5d2844618a8edf311dfac56e5/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:56:45.654277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218e5a4-aad5-4cd1-a175-c2f890785280 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:45.654840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218e5a4-aad5-4cd1-a175-c2f890785280 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:45.654840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218e5a4-aad5-4cd1-a175-c2f890785280 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:45.656177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 552/1085] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:45.661392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aab8c-c445-41da-9936-8c16db7a531b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:45.661812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aab8c-c445-41da-9936-8c16db7a531b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:45.661917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07aab8c-c445-41da-9936-8c16db7a531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:45.679431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f07aab8c-c445-41da-9936-8c16db7a531b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:45.680598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f07aab8c-c445-41da-9936-8c16db7a531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:45.688196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:45.746463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 534/1086] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:45.864578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e496fd-439b-4eed-bc6c-85531e831a0a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:45.864882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e496fd-439b-4eed-bc6c-85531e831a0a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:45.865151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e496fd-439b-4eed-bc6c-85531e831a0a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:45.866491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 553/1087] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:45.872357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb53259-af7e-4c9f-b4c2-e8bad9afe9f4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:45.872646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb53259-af7e-4c9f-b4c2-e8bad9afe9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:45.872920 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb53259-af7e-4c9f-b4c2-e8bad9afe9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:45.890597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-dcb53259-af7e-4c9f-b4c2-e8bad9afe9f4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:45.891800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dcb53259-af7e-4c9f-b4c2-e8bad9afe9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:45.955556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 535/1088] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:46.011738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e496fd-439b-4eed-bc6c-85531e831a0a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.077327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:46.111497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e496fd-439b-4eed-bc6c-85531e831a0a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.145469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bbe05-fb7e-4ecb-8442-9e20f1fb240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.146072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bbe05-fb7e-4ecb-8442-9e20f1fb240f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.146443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bbe05-fb7e-4ecb-8442-9e20f1fb240f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.146709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bbe05-fb7e-4ecb-8442-9e20f1fb240f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.156962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bbe05-fb7e-4ecb-8442-9e20f1fb240f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-890bbe05-fb7e-4ecb-8442-9e20f1fb240f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:46.161513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:46.162583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:46.163480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 554/1089] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:46.168522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb53259-af7e-4c9f-b4c2-e8bad9afe9f4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.218721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bbe05-fb7e-4ecb-8442-9e20f1fb240f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.238734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:46.267398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb53259-af7e-4c9f-b4c2-e8bad9afe9f4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.301859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.302450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.302723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.303020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.303675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:46.303971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:46.305322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:46.313060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 536/1090] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:46.318640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.319610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bbe05-fb7e-4ecb-8442-9e20f1fb240f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.345713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:46.353870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68aa4f-617b-4cb0-b370-daff5cc5877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.358639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68aa4f-617b-4cb0-b370-daff5cc5877c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.358639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68aa4f-617b-4cb0-b370-daff5cc5877c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.358639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68aa4f-617b-4cb0-b370-daff5cc5877c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.365196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68aa4f-617b-4cb0-b370-daff5cc5877c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9e68aa4f-617b-4cb0-b370-daff5cc5877c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:56:46.369848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:46.370905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:56:46.372234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 555/1091] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:46.377174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68aa4f-617b-4cb0-b370-daff5cc5877c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.416887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6736fcb-577d-418b-956e-c51a7494dffb None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.449888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.450497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.450771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.451067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.453068 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:46.462670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:46.467344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:46.477061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68aa4f-617b-4cb0-b370-daff5cc5877c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:46.478086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:46.479046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:46.479956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:46.479956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:46.479956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:46.479956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 537/1092] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:46.485061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.512079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.512666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.512935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.513195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.513839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:46.514131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:46.515509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:46.523368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 556/1093] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:46.529611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.579221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:46.585857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15ae5-cc34-46e2-90e8-8db048be2aa2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.619828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.620431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.620747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.621030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.621667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:46.621970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:46.623384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:46.629558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105abfd-9280-45ee-ad3d-cfd3333cc792 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.635553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 538/1094] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:46.641218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.663737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec220d-b955-490a-8663-57800504c322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.664361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec220d-b955-490a-8663-57800504c322 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.664642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec220d-b955-490a-8663-57800504c322 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.664916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec220d-b955-490a-8663-57800504c322 None admin] PATH_INFO: `/v3/users/0a38dee5d2844618a8edf311dfac56e5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.666549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec220d-b955-490a-8663-57800504c322 None admin] RBAC: Authorizing `identity:get_user(user_id=0a38dee5d2844618a8edf311dfac56e5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:46.668077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec220d-b955-490a-8663-57800504c322 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:46.669649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 557/1095] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/users/0a38dee5d2844618a8edf311dfac56e5 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:46.675502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec220d-b955-490a-8663-57800504c322 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.689557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:46.741349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccf26d-e83b-4e5a-ad95-1c301d9e9039 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.779698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec220d-b955-490a-8663-57800504c322 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.782155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.782599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.782871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.783162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.790747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:46.792232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:46.797870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:56:46.803330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:46.804259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:46.805430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 539/1096] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:46.810846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.815665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f7def-310e-4ab8-84ad-f9dbdf2f4d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.816272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f7def-310e-4ab8-84ad-f9dbdf2f4d69 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.816581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f7def-310e-4ab8-84ad-f9dbdf2f4d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.816871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f7def-310e-4ab8-84ad-f9dbdf2f4d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.886615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f7def-310e-4ab8-84ad-f9dbdf2f4d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:46.888111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f7def-310e-4ab8-84ad-f9dbdf2f4d69 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:46.909251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9f61-84cf-4d5f-85dd-194f0f3e08c7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:46.916226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:46.943192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:46.943794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:46.944080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:46.944347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:46.945067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:46.945369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:46.947439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:46.965990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 540/1097] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:46.971796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:46.989483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 558/1098] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:56:46.995782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f7def-310e-4ab8-84ad-f9dbdf2f4d69 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:47.012636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:47.068952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba247d-f2aa-4a8d-9e2e-6c998ae61db9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:47.092411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f7def-310e-4ab8-84ad-f9dbdf2f4d69 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:47.102211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c45b8a-72ee-4fad-8a0b-21cdc1d63208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:47.102867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c45b8a-72ee-4fad-8a0b-21cdc1d63208 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:47.103173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c45b8a-72ee-4fad-8a0b-21cdc1d63208 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:47.103445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c45b8a-72ee-4fad-8a0b-21cdc1d63208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:47.125880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a824f7-d52b-48ae-aa1b-daae07633e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:47.126494 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a824f7-d52b-48ae-aa1b-daae07633e6f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:47.126770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a824f7-d52b-48ae-aa1b-daae07633e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:47.127072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a824f7-d52b-48ae-aa1b-daae07633e6f None admin] PATH_INFO: `/v3/projects/e460c54fc5204a408e8564962f09c869` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:47.128176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a824f7-d52b-48ae-aa1b-daae07633e6f None admin] RBAC: Authorizing `identity:get_project(project_id=e460c54fc5204a408e8564962f09c869)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:47.129631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a824f7-d52b-48ae-aa1b-daae07633e6f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:47.131281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 559/1099] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/projects/e460c54fc5204a408e8564962f09c869 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:47.133840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:47.137249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a824f7-d52b-48ae-aa1b-daae07633e6f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:47.170656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c45b8a-72ee-4fad-8a0b-21cdc1d63208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:47.172173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c45b8a-72ee-4fad-8a0b-21cdc1d63208 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:47.219792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:47.234356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a824f7-d52b-48ae-aa1b-daae07633e6f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:47.268721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:47.269232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:47.269332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 541/1100] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:56:47.269511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:47.269780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:47.270495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=e460c54fc5204a408e8564962f09c869, user.id=0a38dee5d2844618a8edf311dfac56e5) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:47.270808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:47.272275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:47.275135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c45b8a-72ee-4fad-8a0b-21cdc1d63208 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:47.278172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 560/1101] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:47 2022] GET /identity/v3/role_assignments?user.id=0a38dee5d2844618a8edf311dfac56e5&scope.project.id=e460c54fc5204a408e8564962f09c869&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:47.339653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:47.378588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c45b8a-72ee-4fad-8a0b-21cdc1d63208 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:47.414234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:47.414859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:47.415697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:47.415964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:47.427593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:47.429030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:56:47.443791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:47.444933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:56:47.446239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:56:47.446239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:56:47.446239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:47.446239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 542/1102] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 07:56:47.451207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:47.549075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816616-b7a2-4125-8a9b-112ff092e9df None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:47.582353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:47.582962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:47.583236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:47.583501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:47.584122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:47.584416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:47.585762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:47.597198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 561/1103] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:47.603607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:47.702386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc247f77-ed1e-431b-88af-83a753208c34 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:47.718475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:47.736488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:47.737116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:47.737390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:47.737659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:47.744374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:47.745806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:56:47.757229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:56:47.758145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:56:47.764179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 543/1104] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:56:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:56:47.764704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:47.779600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:47.864837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448f661-f0ab-425c-bc00-512865322389 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:47.896845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d39f23-6716-4d22-b91c-48252e0a46b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:47.897456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d39f23-6716-4d22-b91c-48252e0a46b6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:47.897737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d39f23-6716-4d22-b91c-48252e0a46b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:47.898010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d39f23-6716-4d22-b91c-48252e0a46b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:47.898692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d39f23-6716-4d22-b91c-48252e0a46b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:47.899025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d39f23-6716-4d22-b91c-48252e0a46b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:47.901106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d39f23-6716-4d22-b91c-48252e0a46b6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:47.913120 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 562/1105] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:56:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:56:47.919981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:48.017445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a3c56-b4ac-4bb5-be9a-c06529a6ad54 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:48.049387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53934232-189b-40c6-b32e-bfbc1a424258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:48.049972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53934232-189b-40c6-b32e-bfbc1a424258 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:48.050245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53934232-189b-40c6-b32e-bfbc1a424258 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:48.050511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53934232-189b-40c6-b32e-bfbc1a424258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:48.051249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53934232-189b-40c6-b32e-bfbc1a424258 None admin] RBAC: Adding query filter params (role.id=96417dd6dfef4a79b4cb31119e1cbc94, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=03cc4435b9b0444b9f278e15398443c7) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:48.051552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53934232-189b-40c6-b32e-bfbc1a424258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:48.052954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53934232-189b-40c6-b32e-bfbc1a424258 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:48.058483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 544/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:47 2022] GET /identity/v3/role_assignments?user.id=03cc4435b9b0444b9f278e15398443c7&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=96417dd6dfef4a79b4cb31119e1cbc94 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:49.216892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273f479-7535-4940-aacb-158c346c0f3b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:49.217398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273f479-7535-4940-aacb-158c346c0f3b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:49.217510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273f479-7535-4940-aacb-158c346c0f3b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:49.218766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 563/1107] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:49.224023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306b5d-157b-47f3-8074-70234fde80a7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:49.224442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306b5d-157b-47f3-8074-70234fde80a7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:49.224563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306b5d-157b-47f3-8074-70234fde80a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:49.242696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fa306b5d-157b-47f3-8074-70234fde80a7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:49.243934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fa306b5d-157b-47f3-8074-70234fde80a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:49.276474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:49.308788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 545/1108] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:49.574879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273f479-7535-4940-aacb-158c346c0f3b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:49.643040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:49.672794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273f479-7535-4940-aacb-158c346c0f3b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:49.705656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d1d41-9397-46ac-866c-75eb53f37002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:49.706261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1d41-9397-46ac-866c-75eb53f37002 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:49.706533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1d41-9397-46ac-866c-75eb53f37002 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:49.706797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1d41-9397-46ac-866c-75eb53f37002 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:49.707916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d1d41-9397-46ac-866c-75eb53f37002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:49.709330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d1d41-9397-46ac-866c-75eb53f37002 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:49.710861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 564/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:56:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:49.716673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa306b5d-157b-47f3-8074-70234fde80a7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:49.813528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa306b5d-157b-47f3-8074-70234fde80a7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:49.846366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c6a0e-e7a0-4c84-873b-f712aeb856dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:49.846895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c6a0e-e7a0-4c84-873b-f712aeb856dc None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:49.847196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c6a0e-e7a0-4c84-873b-f712aeb856dc None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:49.847460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c6a0e-e7a0-4c84-873b-f712aeb856dc None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:49.848096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c6a0e-e7a0-4c84-873b-f712aeb856dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:49.849497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c6a0e-e7a0-4c84-873b-f712aeb856dc None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:49.952991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 546/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:56:49 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:56:51.889996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab594ccf-a59a-460f-b1ca-abf2e6d96050 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:51.890510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab594ccf-a59a-460f-b1ca-abf2e6d96050 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:51.890599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab594ccf-a59a-460f-b1ca-abf2e6d96050 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:51.892068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 565/1111] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:51.897359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3592-96f9-4d9f-a223-86a9c7908978 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:51.897762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3592-96f9-4d9f-a223-86a9c7908978 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:51.897907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3592-96f9-4d9f-a223-86a9c7908978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:51.916202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ed9f3592-96f9-4d9f-a223-86a9c7908978 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:51.917391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ed9f3592-96f9-4d9f-a223-86a9c7908978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:51.937129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:51.986207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 547/1112] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:52.256217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab594ccf-a59a-460f-b1ca-abf2e6d96050 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:52.367496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab594ccf-a59a-460f-b1ca-abf2e6d96050 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:52.400066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c61b0e-4ff1-4fba-a286-bd787e05223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:52.400673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61b0e-4ff1-4fba-a286-bd787e05223a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:52.400948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61b0e-4ff1-4fba-a286-bd787e05223a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:52.401213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c61b0e-4ff1-4fba-a286-bd787e05223a None admin] PATH_INFO: `/v3/roles/af615e7795ae42019a45dfe2bb205c49` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:52.402594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c61b0e-4ff1-4fba-a286-bd787e05223a None admin] RBAC: Authorizing `identity:get_role(role_id=af615e7795ae42019a45dfe2bb205c49)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:52.403981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c61b0e-4ff1-4fba-a286-bd787e05223a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:52.405289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 566/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:52 2022] GET /identity/v3/roles/af615e7795ae42019a45dfe2bb205c49 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:52.411277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f3592-96f9-4d9f-a223-86a9c7908978 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:52.507122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f3592-96f9-4d9f-a223-86a9c7908978 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:52.540162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd7021-50f3-417a-8d1a-21aec20cbc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:52.540767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd7021-50f3-417a-8d1a-21aec20cbc97 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:52.541041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd7021-50f3-417a-8d1a-21aec20cbc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:52.541309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd7021-50f3-417a-8d1a-21aec20cbc97 None admin] PATH_INFO: `/v3/users/c8db6088b2dc4be69215ac86b00bfbef` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:52.553869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd7021-50f3-417a-8d1a-21aec20cbc97 None admin] RBAC: Authorizing `identity:get_user(user_id=c8db6088b2dc4be69215ac86b00bfbef)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:52.555371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd7021-50f3-417a-8d1a-21aec20cbc97 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:52.562069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 548/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:52 2022] GET /identity/v3/users/c8db6088b2dc4be69215ac86b00bfbef => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:52.562522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c61b0e-4ff1-4fba-a286-bd787e05223a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:52.571773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:52.662660 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c61b0e-4ff1-4fba-a286-bd787e05223a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:52.695379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bd375-0ec9-4bbd-9b33-3ae0863fa205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:52.695968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bd375-0ec9-4bbd-9b33-3ae0863fa205 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:52.696268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bd375-0ec9-4bbd-9b33-3ae0863fa205 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:52.696536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bd375-0ec9-4bbd-9b33-3ae0863fa205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:52.761269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bd375-0ec9-4bbd-9b33-3ae0863fa205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:52.762703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bd375-0ec9-4bbd-9b33-3ae0863fa205 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:52.771083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:52.859862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 567/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:56:52.865916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd7021-50f3-417a-8d1a-21aec20cbc97 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:52.881388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:52.963547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd7021-50f3-417a-8d1a-21aec20cbc97 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:52.997041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8498d-9eb2-4403-85e8-8663450eaf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:52.997651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8498d-9eb2-4403-85e8-8663450eaf9b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:52.997926 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8498d-9eb2-4403-85e8-8663450eaf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:52.998197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8498d-9eb2-4403-85e8-8663450eaf9b None admin] PATH_INFO: `/v3/projects/e460c54fc5204a408e8564962f09c869` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:52.999357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8498d-9eb2-4403-85e8-8663450eaf9b None admin] RBAC: Authorizing `identity:get_project(project_id=e460c54fc5204a408e8564962f09c869)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:53.000780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8498d-9eb2-4403-85e8-8663450eaf9b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:53.002388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 549/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:56:52 2022] GET /identity/v3/projects/e460c54fc5204a408e8564962f09c869 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:53.008533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bd375-0ec9-4bbd-9b33-3ae0863fa205 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:53.104329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bd375-0ec9-4bbd-9b33-3ae0863fa205 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:53.111284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:53.136348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303c22b-3bbb-4eb6-9793-2733b3011eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:53.137000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c22b-3bbb-4eb6-9793-2733b3011eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:53.137265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c22b-3bbb-4eb6-9793-2733b3011eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:53.137535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303c22b-3bbb-4eb6-9793-2733b3011eb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:53.138242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303c22b-3bbb-4eb6-9793-2733b3011eb5 None admin] RBAC: Adding query filter params (role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=e460c54fc5204a408e8564962f09c869, user.id=c8db6088b2dc4be69215ac86b00bfbef) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:53.138546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303c22b-3bbb-4eb6-9793-2733b3011eb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:53.139986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303c22b-3bbb-4eb6-9793-2733b3011eb5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:53.145068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 568/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:53 2022] GET /identity/v3/role_assignments?user.id=c8db6088b2dc4be69215ac86b00bfbef&scope.project.id=e460c54fc5204a408e8564962f09c869&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:55.008576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec365b6f-3d13-4564-87c1-0b904f695f5e None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:55.009152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec365b6f-3d13-4564-87c1-0b904f695f5e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:55.009152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec365b6f-3d13-4564-87c1-0b904f695f5e None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:55.010484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 550/1118] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:55.015752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d90b-b3d0-4d01-a60d-77eab8c85fcb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:55.016221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d90b-b3d0-4d01-a60d-77eab8c85fcb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:55.016353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d90b-b3d0-4d01-a60d-77eab8c85fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:55.033976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7c86d90b-b3d0-4d01-a60d-77eab8c85fcb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:55.035139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c86d90b-b3d0-4d01-a60d-77eab8c85fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:55.100156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 569/1119] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:55.363768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec365b6f-3d13-4564-87c1-0b904f695f5e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:55.443601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:55.461831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec365b6f-3d13-4564-87c1-0b904f695f5e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:55.495475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f08c7-84bb-43d9-a48f-543bad6cb3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:55.496132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f08c7-84bb-43d9-a48f-543bad6cb3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:55.496394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f08c7-84bb-43d9-a48f-543bad6cb3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:55.496666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f08c7-84bb-43d9-a48f-543bad6cb3d2 None admin] PATH_INFO: `/v3/roles/af615e7795ae42019a45dfe2bb205c49` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:55.498054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f08c7-84bb-43d9-a48f-543bad6cb3d2 None admin] RBAC: Authorizing `identity:get_role(role_id=af615e7795ae42019a45dfe2bb205c49)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:55.499475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f08c7-84bb-43d9-a48f-543bad6cb3d2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:55.500783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 551/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:55 2022] GET /identity/v3/roles/af615e7795ae42019a45dfe2bb205c49 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:55.506488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86d90b-b3d0-4d01-a60d-77eab8c85fcb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:55.598822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:55.601704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86d90b-b3d0-4d01-a60d-77eab8c85fcb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:55.634905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dbc9e-7374-4f99-afc1-f59f902e9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:55.635489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dbc9e-7374-4f99-afc1-f59f902e9972 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:55.635763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dbc9e-7374-4f99-afc1-f59f902e9972 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:55.636030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dbc9e-7374-4f99-afc1-f59f902e9972 None admin] PATH_INFO: `/v3/users/c8db6088b2dc4be69215ac86b00bfbef` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:55.637173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dbc9e-7374-4f99-afc1-f59f902e9972 None admin] RBAC: Authorizing `identity:get_user(user_id=c8db6088b2dc4be69215ac86b00bfbef)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:55.638998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dbc9e-7374-4f99-afc1-f59f902e9972 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:55.643010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 570/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:55 2022] GET /identity/v3/users/c8db6088b2dc4be69215ac86b00bfbef => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:55.646287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f08c7-84bb-43d9-a48f-543bad6cb3d2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:55.741424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f08c7-84bb-43d9-a48f-543bad6cb3d2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:55.773816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b390717-ba31-4563-b001-a77dd1ab0774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:55.774440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390717-ba31-4563-b001-a77dd1ab0774 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:55.774713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390717-ba31-4563-b001-a77dd1ab0774 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:55.775001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390717-ba31-4563-b001-a77dd1ab0774 None admin] PATH_INFO: `/v3/projects/e460c54fc5204a408e8564962f09c869` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:55.776086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b390717-ba31-4563-b001-a77dd1ab0774 None admin] RBAC: Authorizing `identity:get_project(project_id=e460c54fc5204a408e8564962f09c869)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:55.777498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b390717-ba31-4563-b001-a77dd1ab0774 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:55.779099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 552/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:56:55 2022] GET /identity/v3/projects/e460c54fc5204a408e8564962f09c869 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:55.785085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dbc9e-7374-4f99-afc1-f59f902e9972 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:55.882144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dbc9e-7374-4f99-afc1-f59f902e9972 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:55.914825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14340d-6225-4567-85e8-ca25dabb7fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:55.915490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14340d-6225-4567-85e8-ca25dabb7fd1 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:55.915775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14340d-6225-4567-85e8-ca25dabb7fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:55.916047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14340d-6225-4567-85e8-ca25dabb7fd1 None admin] PATH_INFO: `/v3/projects/e460c54fc5204a408e8564962f09c869/users/c8db6088b2dc4be69215ac86b00bfbef/roles/af615e7795ae42019a45dfe2bb205c49` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:55.918190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14340d-6225-4567-85e8-ca25dabb7fd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e460c54fc5204a408e8564962f09c869, user_id=c8db6088b2dc4be69215ac86b00bfbef, role_id=af615e7795ae42019a45dfe2bb205c49)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:55.919655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14340d-6225-4567-85e8-ca25dabb7fd1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:55.998389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 571/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:56:55 2022] PUT /identity/v3/projects/e460c54fc5204a408e8564962f09c869/users/c8db6088b2dc4be69215ac86b00bfbef/roles/af615e7795ae42019a45dfe2bb205c49 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:56:57.861651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8048-2f0c-414f-a825-572cbb646adb None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:57.862169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8048-2f0c-414f-a825-572cbb646adb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:57.862261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8048-2f0c-414f-a825-572cbb646adb None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:57.863601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 553/1124] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:56:57.868725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e98cb-687d-4665-8141-6814973ae77d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:57.869162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e98cb-687d-4665-8141-6814973ae77d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:57.869272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e98cb-687d-4665-8141-6814973ae77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:57.887190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3b5e98cb-687d-4665-8141-6814973ae77d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:56:57.888353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3b5e98cb-687d-4665-8141-6814973ae77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:56:57.910568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:57.953069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 572/1125] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:56:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:56:58.223685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e8048-2f0c-414f-a825-572cbb646adb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:58.243981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:58.324753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e8048-2f0c-414f-a825-572cbb646adb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:58.356795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2e6cc-bec7-4f33-9f6a-284a5c2a16aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:58.357406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2e6cc-bec7-4f33-9f6a-284a5c2a16aa None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:58.357686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2e6cc-bec7-4f33-9f6a-284a5c2a16aa None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:58.357959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2e6cc-bec7-4f33-9f6a-284a5c2a16aa None admin] PATH_INFO: `/v3/roles/af615e7795ae42019a45dfe2bb205c49` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:58.359374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2e6cc-bec7-4f33-9f6a-284a5c2a16aa None admin] RBAC: Authorizing `identity:get_role(role_id=af615e7795ae42019a45dfe2bb205c49)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:58.360739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2e6cc-bec7-4f33-9f6a-284a5c2a16aa None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:58.362028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 554/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:58 2022] GET /identity/v3/roles/af615e7795ae42019a45dfe2bb205c49 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:58.368305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e98cb-687d-4665-8141-6814973ae77d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:58.464203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e98cb-687d-4665-8141-6814973ae77d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:58.497045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09040-83c4-4a0d-bf35-780235bb3a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:58.497591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09040-83c4-4a0d-bf35-780235bb3a81 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:58.497863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09040-83c4-4a0d-bf35-780235bb3a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:58.498129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e09040-83c4-4a0d-bf35-780235bb3a81 None admin] PATH_INFO: `/v3/users/c8db6088b2dc4be69215ac86b00bfbef` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:58.499249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e09040-83c4-4a0d-bf35-780235bb3a81 None admin] RBAC: Authorizing `identity:get_user(user_id=c8db6088b2dc4be69215ac86b00bfbef)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:58.500655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e09040-83c4-4a0d-bf35-780235bb3a81 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:58.502195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 573/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:56:58 2022] GET /identity/v3/users/c8db6088b2dc4be69215ac86b00bfbef => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:58.507931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2e6cc-bec7-4f33-9f6a-284a5c2a16aa None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:58.589930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:58.604268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2e6cc-bec7-4f33-9f6a-284a5c2a16aa None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:58.637228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e087b-e65a-415c-83bc-a971599cb107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:58.637818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e087b-e65a-415c-83bc-a971599cb107 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:58.638094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e087b-e65a-415c-83bc-a971599cb107 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:58.638368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e087b-e65a-415c-83bc-a971599cb107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:58.702050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e087b-e65a-415c-83bc-a971599cb107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:58.703506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e087b-e65a-415c-83bc-a971599cb107 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:58.782677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:58.797601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 555/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:56:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:56:58.803534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09040-83c4-4a0d-bf35-780235bb3a81 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:58.831386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:56:58.900589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e09040-83c4-4a0d-bf35-780235bb3a81 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:58.933084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c1ed8-c705-4472-a18c-6df7925aca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:58.933690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c1ed8-c705-4472-a18c-6df7925aca01 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:58.933964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c1ed8-c705-4472-a18c-6df7925aca01 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:58.934229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c1ed8-c705-4472-a18c-6df7925aca01 None admin] PATH_INFO: `/v3/projects/e460c54fc5204a408e8564962f09c869` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:58.935359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c1ed8-c705-4472-a18c-6df7925aca01 None admin] RBAC: Authorizing `identity:get_project(project_id=e460c54fc5204a408e8564962f09c869)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:58.936778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c1ed8-c705-4472-a18c-6df7925aca01 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:58.938364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 574/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:56:58 2022] GET /identity/v3/projects/e460c54fc5204a408e8564962f09c869 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:56:58.944309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e087b-e65a-415c-83bc-a971599cb107 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:56:59.039449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e087b-e65a-415c-83bc-a971599cb107 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:56:59.072555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb322f37-9ecc-4798-b9e4-db66f43da27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:56:59.073398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb322f37-9ecc-4798-b9e4-db66f43da27c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:56:59.073675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb322f37-9ecc-4798-b9e4-db66f43da27c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:56:59.073947 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb322f37-9ecc-4798-b9e4-db66f43da27c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:56:59.074661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb322f37-9ecc-4798-b9e4-db66f43da27c None admin] RBAC: Adding query filter params (role.id=af615e7795ae42019a45dfe2bb205c49, scope.project.id=e460c54fc5204a408e8564962f09c869, user.id=c8db6088b2dc4be69215ac86b00bfbef) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:56:59.074985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb322f37-9ecc-4798-b9e4-db66f43da27c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:56:59.076396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb322f37-9ecc-4798-b9e4-db66f43da27c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:56:59.081848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 556/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:56:58 2022] GET /identity/v3/role_assignments?user.id=c8db6088b2dc4be69215ac86b00bfbef&scope.project.id=e460c54fc5204a408e8564962f09c869&role.id=af615e7795ae42019a45dfe2bb205c49 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:00.982273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504728a-eb84-4c1e-b62a-4436cb2eba8e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:00.982800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504728a-eb84-4c1e-b62a-4436cb2eba8e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:00.982800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504728a-eb84-4c1e-b62a-4436cb2eba8e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:00.984181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 575/1131] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:00.989321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe322f-2a1b-423c-a175-39919fb139d2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:00.989706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe322f-2a1b-423c-a175-39919fb139d2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:00.989863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe322f-2a1b-423c-a175-39919fb139d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:01.007987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ebfe322f-2a1b-423c-a175-39919fb139d2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:01.009204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ebfe322f-2a1b-423c-a175-39919fb139d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:01.053472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:01.073775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 557/1132] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:01.340271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504728a-eb84-4c1e-b62a-4436cb2eba8e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:01.431263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:01.438516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504728a-eb84-4c1e-b62a-4436cb2eba8e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:01.471171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2513d9-5e9d-4fc1-ba68-9298ed4ca538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:01.471790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2513d9-5e9d-4fc1-ba68-9298ed4ca538 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:01.472069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2513d9-5e9d-4fc1-ba68-9298ed4ca538 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:01.472336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2513d9-5e9d-4fc1-ba68-9298ed4ca538 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:01.473474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2513d9-5e9d-4fc1-ba68-9298ed4ca538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:01.474897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2513d9-5e9d-4fc1-ba68-9298ed4ca538 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:01.476458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 576/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:57:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:01.482746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe322f-2a1b-423c-a175-39919fb139d2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:01.579673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe322f-2a1b-423c-a175-39919fb139d2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:01.613091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6373ee9-217f-495d-af36-386e7f791c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:01.613626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6373ee9-217f-495d-af36-386e7f791c56 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:01.613904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6373ee9-217f-495d-af36-386e7f791c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:01.614175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6373ee9-217f-495d-af36-386e7f791c56 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:01.614806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6373ee9-217f-495d-af36-386e7f791c56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:01.616261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6373ee9-217f-495d-af36-386e7f791c56 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:01.675288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 558/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:57:01 2022] POST /identity/v3/projects => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:57:03.597446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da1956-c695-47ec-90ef-99cc444c7473 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:03.597998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da1956-c695-47ec-90ef-99cc444c7473 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:03.597998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da1956-c695-47ec-90ef-99cc444c7473 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:03.599369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 577/1135] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:03.604733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc4d72-b12e-4d4d-b84f-23ce7d753c71 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:03.605166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc4d72-b12e-4d4d-b84f-23ce7d753c71 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:03.605287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc4d72-b12e-4d4d-b84f-23ce7d753c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:03.623548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5ddc4d72-b12e-4d4d-b84f-23ce7d753c71 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:03.624724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5ddc4d72-b12e-4d4d-b84f-23ce7d753c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:03.650086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:03.689627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 559/1136] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:03.964719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da1956-c695-47ec-90ef-99cc444c7473 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:04.062955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da1956-c695-47ec-90ef-99cc444c7473 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:04.096033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed354a-4c23-477a-acff-c14f79771158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:04.096637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed354a-4c23-477a-acff-c14f79771158 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:04.096913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed354a-4c23-477a-acff-c14f79771158 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:04.097182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed354a-4c23-477a-acff-c14f79771158 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:04.098279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed354a-4c23-477a-acff-c14f79771158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:04.099731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed354a-4c23-477a-acff-c14f79771158 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:04.101279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 578/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:57:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:04.107433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc4d72-b12e-4d4d-b84f-23ce7d753c71 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:04.201826 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc4d72-b12e-4d4d-b84f-23ce7d753c71 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:04.234613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134f7e6-a70a-47eb-a501-40f13a3c8c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:04.235163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f7e6-a70a-47eb-a501-40f13a3c8c84 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:04.235438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f7e6-a70a-47eb-a501-40f13a3c8c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:04.235705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f7e6-a70a-47eb-a501-40f13a3c8c84 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:04.236341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f7e6-a70a-47eb-a501-40f13a3c8c84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:04.237751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f7e6-a70a-47eb-a501-40f13a3c8c84 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:04.326056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 560/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:57:04 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 07:57:06.256864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf98f-ecb8-4079-b581-1c6a2ae29d10 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:06.257414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf98f-ecb8-4079-b581-1c6a2ae29d10 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:06.257508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf98f-ecb8-4079-b581-1c6a2ae29d10 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:06.258777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 579/1139] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:06.264012 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808b7d-0fe0-4678-9a13-c7d5eb4306e3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:06.264429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808b7d-0fe0-4678-9a13-c7d5eb4306e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:06.264565 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808b7d-0fe0-4678-9a13-c7d5eb4306e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:06.282141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a8808b7d-0fe0-4678-9a13-c7d5eb4306e3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:06.283354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a8808b7d-0fe0-4678-9a13-c7d5eb4306e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:06.297993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:06.349934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 561/1140] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:06.619284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bf98f-ecb8-4079-b581-1c6a2ae29d10 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:06.648662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:06.717008 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bf98f-ecb8-4079-b581-1c6a2ae29d10 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:06.749114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb1a52-2998-483e-839c-d60dbe32ea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:06.749712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb1a52-2998-483e-839c-d60dbe32ea53 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:06.749983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb1a52-2998-483e-839c-d60dbe32ea53 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:06.750248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb1a52-2998-483e-839c-d60dbe32ea53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:06.762075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb1a52-2998-483e-839c-d60dbe32ea53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4fdb1a52-2998-483e-839c-d60dbe32ea53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:57:06.766653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:57:06.767850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:57:06.768987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 580/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:57:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:57:06.774441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8808b7d-0fe0-4678-9a13-c7d5eb4306e3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:06.871520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8808b7d-0fe0-4678-9a13-c7d5eb4306e3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:06.904837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:06.905426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:06.905704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:06.905974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:06.906625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:06.906925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:06.908304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:06.916084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 562/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:57:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:06.921765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb1a52-2998-483e-839c-d60dbe32ea53 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:07.008610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:07.017577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb1a52-2998-483e-839c-d60dbe32ea53 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:07.050071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ce3f5-6267-44fa-a8a8-9388a7fa12b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:07.050667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ce3f5-6267-44fa-a8a8-9388a7fa12b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:07.050970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ce3f5-6267-44fa-a8a8-9388a7fa12b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:07.051390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ce3f5-6267-44fa-a8a8-9388a7fa12b2 None admin] PATH_INFO: `/v3/users/8520f57090104daf92e913966663775f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:07.064867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ce3f5-6267-44fa-a8a8-9388a7fa12b2 None admin] RBAC: Authorizing `identity:get_user(user_id=8520f57090104daf92e913966663775f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:07.066292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ce3f5-6267-44fa-a8a8-9388a7fa12b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:07.067865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 581/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:57:06 2022] GET /identity/v3/users/8520f57090104daf92e913966663775f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:07.074593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:07.087005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:07.170963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9a77-93d0-45b4-8063-3b973f007292 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:07.204257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527d45a-66a1-49e0-aa20-172f243a81d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:07.204848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527d45a-66a1-49e0-aa20-172f243a81d9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:07.205128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527d45a-66a1-49e0-aa20-172f243a81d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:07.205400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527d45a-66a1-49e0-aa20-172f243a81d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:07.269971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527d45a-66a1-49e0-aa20-172f243a81d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:07.271439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527d45a-66a1-49e0-aa20-172f243a81d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:07.282380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:07.365804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 563/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:57:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:57:07.372080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ce3f5-6267-44fa-a8a8-9388a7fa12b2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:07.470012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ce3f5-6267-44fa-a8a8-9388a7fa12b2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:07.505060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ae366-dc86-4823-a2de-742c9ebd30f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:07.505684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae366-dc86-4823-a2de-742c9ebd30f9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:07.505962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae366-dc86-4823-a2de-742c9ebd30f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:07.506238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae366-dc86-4823-a2de-742c9ebd30f9 None admin] PATH_INFO: `/v3/projects/6a92499b689042dc9ff54184c3f70375` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:07.507398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ae366-dc86-4823-a2de-742c9ebd30f9 None admin] RBAC: Authorizing `identity:get_project(project_id=6a92499b689042dc9ff54184c3f70375)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:07.508829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ae366-dc86-4823-a2de-742c9ebd30f9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:07.510507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 582/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:57:07 2022] GET /identity/v3/projects/6a92499b689042dc9ff54184c3f70375 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:07.516713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527d45a-66a1-49e0-aa20-172f243a81d9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:07.616292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527d45a-66a1-49e0-aa20-172f243a81d9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:07.627623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:07.650515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42a8aa-e2f7-4154-94b2-aa709e61c89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:07.651157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42a8aa-e2f7-4154-94b2-aa709e61c89c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:07.651433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42a8aa-e2f7-4154-94b2-aa709e61c89c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:07.651706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42a8aa-e2f7-4154-94b2-aa709e61c89c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:07.652429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42a8aa-e2f7-4154-94b2-aa709e61c89c None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=6a92499b689042dc9ff54184c3f70375, user.id=8520f57090104daf92e913966663775f) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:07.652739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42a8aa-e2f7-4154-94b2-aa709e61c89c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:07.654168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42a8aa-e2f7-4154-94b2-aa709e61c89c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:07.660033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 564/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:57:07 2022] GET /identity/v3/role_assignments?user.id=8520f57090104daf92e913966663775f&scope.project.id=6a92499b689042dc9ff54184c3f70375&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:09.531391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e18add-25b5-4b5c-b62e-71b6bc998f8a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:09.531946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e18add-25b5-4b5c-b62e-71b6bc998f8a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:09.531946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e18add-25b5-4b5c-b62e-71b6bc998f8a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:09.533245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 583/1147] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:09.538500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71c99e-8155-4b90-9096-0fc2beee4af0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:09.538901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71c99e-8155-4b90-9096-0fc2beee4af0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:09.539111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71c99e-8155-4b90-9096-0fc2beee4af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:09.557373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ca71c99e-8155-4b90-9096-0fc2beee4af0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:09.558588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ca71c99e-8155-4b90-9096-0fc2beee4af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:09.623734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 565/1148] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:09.888038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e18add-25b5-4b5c-b62e-71b6bc998f8a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:09.905806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:09.987128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e18add-25b5-4b5c-b62e-71b6bc998f8a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:10.019409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b10ae-4bed-4282-bdd5-92aaf7875b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:10.020019 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b10ae-4bed-4282-bdd5-92aaf7875b70 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:10.020297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b10ae-4bed-4282-bdd5-92aaf7875b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:10.020564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b10ae-4bed-4282-bdd5-92aaf7875b70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:10.030804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b10ae-4bed-4282-bdd5-92aaf7875b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6e3b10ae-4bed-4282-bdd5-92aaf7875b70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:57:10.035513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:57:10.036591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:57:10.037454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 584/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:57:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:57:10.042511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71c99e-8155-4b90-9096-0fc2beee4af0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:10.138919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71c99e-8155-4b90-9096-0fc2beee4af0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:10.141158 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:10.171819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:10.172416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:10.172700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:10.172969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:10.173622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:10.173920 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:10.175302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:10.183875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 566/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:57:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:10.189702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b10ae-4bed-4282-bdd5-92aaf7875b70 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:10.264536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:10.285826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b10ae-4bed-4282-bdd5-92aaf7875b70 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:10.318481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b970a0-8704-4eca-8a53-d7d8e5de52a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:10.319106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b970a0-8704-4eca-8a53-d7d8e5de52a1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:10.319380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b970a0-8704-4eca-8a53-d7d8e5de52a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:10.319644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b970a0-8704-4eca-8a53-d7d8e5de52a1 None admin] PATH_INFO: `/v3/users/8520f57090104daf92e913966663775f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:10.321327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b970a0-8704-4eca-8a53-d7d8e5de52a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8520f57090104daf92e913966663775f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:10.322735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b970a0-8704-4eca-8a53-d7d8e5de52a1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:10.324585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 585/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:57:10 2022] GET /identity/v3/users/8520f57090104daf92e913966663775f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:10.330444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:10.426015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7d0f-ac40-4348-8e9d-0d46288aa46c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:10.458933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b0adb-7829-41bb-b92c-80974ce62a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:10.459562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0adb-7829-41bb-b92c-80974ce62a09 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:10.459839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0adb-7829-41bb-b92c-80974ce62a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:10.460108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0adb-7829-41bb-b92c-80974ce62a09 None admin] PATH_INFO: `/v3/projects/6a92499b689042dc9ff54184c3f70375` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:10.461240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b0adb-7829-41bb-b92c-80974ce62a09 None admin] RBAC: Authorizing `identity:get_project(project_id=6a92499b689042dc9ff54184c3f70375)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:10.462650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b0adb-7829-41bb-b92c-80974ce62a09 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:10.464262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 567/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:57:10 2022] GET /identity/v3/projects/6a92499b689042dc9ff54184c3f70375 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:10.470078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b970a0-8704-4eca-8a53-d7d8e5de52a1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:10.564775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b970a0-8704-4eca-8a53-d7d8e5de52a1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:10.597624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a90773-5f3c-4557-962b-52eb43d77b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:10.598255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a90773-5f3c-4557-962b-52eb43d77b73 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:10.598530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a90773-5f3c-4557-962b-52eb43d77b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:10.598797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a90773-5f3c-4557-962b-52eb43d77b73 None admin] PATH_INFO: `/v3/projects/6a92499b689042dc9ff54184c3f70375/users/8520f57090104daf92e913966663775f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:10.600676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a90773-5f3c-4557-962b-52eb43d77b73 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a92499b689042dc9ff54184c3f70375, user_id=8520f57090104daf92e913966663775f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:10.602636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a90773-5f3c-4557-962b-52eb43d77b73 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:10.697493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 586/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:57:10 2022] PUT /identity/v3/projects/6a92499b689042dc9ff54184c3f70375/users/8520f57090104daf92e913966663775f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 07:57:12.573784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff155-c3f4-42ec-9693-11eccd39da29 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:12.574308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff155-c3f4-42ec-9693-11eccd39da29 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:12.574308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff155-c3f4-42ec-9693-11eccd39da29 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:12.575719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 568/1154] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:12.580823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f542e1-2d0d-4a9e-b0cf-571befb35f9d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:12.581198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f542e1-2d0d-4a9e-b0cf-571befb35f9d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:12.581359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f542e1-2d0d-4a9e-b0cf-571befb35f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:12.589091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:12.599733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-98f542e1-2d0d-4a9e-b0cf-571befb35f9d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:12.600919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-98f542e1-2d0d-4a9e-b0cf-571befb35f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:12.668333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 587/1155] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:57:12.933440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bff155-c3f4-42ec-9693-11eccd39da29 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:12.970064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:13.034562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bff155-c3f4-42ec-9693-11eccd39da29 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:13.066993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0954627-a2ca-4595-bc1a-3489c7bf6cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:13.067603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0954627-a2ca-4595-bc1a-3489c7bf6cac None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:13.067876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0954627-a2ca-4595-bc1a-3489c7bf6cac None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:13.068146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0954627-a2ca-4595-bc1a-3489c7bf6cac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:13.079099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0954627-a2ca-4595-bc1a-3489c7bf6cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c0954627-a2ca-4595-bc1a-3489c7bf6cac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:57:13.083624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:57:13.084749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:57:13.086472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 569/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:57:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:57:13.091278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f542e1-2d0d-4a9e-b0cf-571befb35f9d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:13.184878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f542e1-2d0d-4a9e-b0cf-571befb35f9d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:13.217571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:13.218160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:13.218436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:13.218707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:13.219384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:13.219687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:13.221039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:13.222333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:13.229790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 588/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:57:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:13.235562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0954627-a2ca-4595-bc1a-3489c7bf6cac None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:13.327238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:13.333646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0954627-a2ca-4595-bc1a-3489c7bf6cac None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:13.365905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe8737-865c-4c0e-b62e-52864c75c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:13.366515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe8737-865c-4c0e-b62e-52864c75c198 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:13.366790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe8737-865c-4c0e-b62e-52864c75c198 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:13.367603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe8737-865c-4c0e-b62e-52864c75c198 None admin] PATH_INFO: `/v3/users/8520f57090104daf92e913966663775f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:13.368737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe8737-865c-4c0e-b62e-52864c75c198 None admin] RBAC: Authorizing `identity:get_user(user_id=8520f57090104daf92e913966663775f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:13.370185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe8737-865c-4c0e-b62e-52864c75c198 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:13.371759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 570/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:57:13 2022] GET /identity/v3/users/8520f57090104daf92e913966663775f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:13.377497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:13.474021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e091aa-837e-476c-a651-e6d296a69268 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:13.507022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772526-3400-41ba-bbe2-bf66ada2e566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:13.507629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772526-3400-41ba-bbe2-bf66ada2e566 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:13.507909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772526-3400-41ba-bbe2-bf66ada2e566 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:13.508182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772526-3400-41ba-bbe2-bf66ada2e566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:13.566499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:13.572700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772526-3400-41ba-bbe2-bf66ada2e566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:13.574432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772526-3400-41ba-bbe2-bf66ada2e566 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:13.669673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 589/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:57:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:57:13.675875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe8737-865c-4c0e-b62e-52864c75c198 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:13.772980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe8737-865c-4c0e-b62e-52864c75c198 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:13.805936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19d4ec-c3eb-43fd-9811-16aac16f9a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:13.806546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d4ec-c3eb-43fd-9811-16aac16f9a8d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:13.806822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d4ec-c3eb-43fd-9811-16aac16f9a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:13.807125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d4ec-c3eb-43fd-9811-16aac16f9a8d None admin] PATH_INFO: `/v3/projects/6a92499b689042dc9ff54184c3f70375` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:13.808224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d4ec-c3eb-43fd-9811-16aac16f9a8d None admin] RBAC: Authorizing `identity:get_project(project_id=6a92499b689042dc9ff54184c3f70375)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:13.809649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d4ec-c3eb-43fd-9811-16aac16f9a8d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:13.811316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 571/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:57:13 2022] GET /identity/v3/projects/6a92499b689042dc9ff54184c3f70375 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:13.817357 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772526-3400-41ba-bbe2-bf66ada2e566 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:13.907734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:13.913413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772526-3400-41ba-bbe2-bf66ada2e566 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:13.946698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a743e8-6366-4d2d-a38c-565cfd4a2f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:13.947309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a743e8-6366-4d2d-a38c-565cfd4a2f6e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:13.947589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a743e8-6366-4d2d-a38c-565cfd4a2f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:13.947861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a743e8-6366-4d2d-a38c-565cfd4a2f6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:13.948833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a743e8-6366-4d2d-a38c-565cfd4a2f6e None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=6a92499b689042dc9ff54184c3f70375, user.id=8520f57090104daf92e913966663775f) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:13.949139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a743e8-6366-4d2d-a38c-565cfd4a2f6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:13.950555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a743e8-6366-4d2d-a38c-565cfd4a2f6e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:13.956149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 590/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:57:13 2022] GET /identity/v3/role_assignments?user.id=8520f57090104daf92e913966663775f&scope.project.id=6a92499b689042dc9ff54184c3f70375&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:15.846863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57529d84-0ae0-4cdd-b294-844c6648d369 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:15.847344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57529d84-0ae0-4cdd-b294-844c6648d369 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:15.847433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57529d84-0ae0-4cdd-b294-844c6648d369 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:15.848771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 572/1162] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:15.854041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2e33e-d80c-4281-88f7-f1e71555b06c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:15.854482 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2e33e-d80c-4281-88f7-f1e71555b06c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:15.854575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2e33e-d80c-4281-88f7-f1e71555b06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:15.872210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-89c2e33e-d80c-4281-88f7-f1e71555b06c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:15.873400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-89c2e33e-d80c-4281-88f7-f1e71555b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:15.937837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 591/1163] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:16.199907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57529d84-0ae0-4cdd-b294-844c6648d369 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:16.231101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:16.299897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57529d84-0ae0-4cdd-b294-844c6648d369 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:16.333294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2cd0f-c28d-4622-8988-4a6396b7e740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:16.333879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2cd0f-c28d-4622-8988-4a6396b7e740 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:16.334156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2cd0f-c28d-4622-8988-4a6396b7e740 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:16.334423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2cd0f-c28d-4622-8988-4a6396b7e740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:16.400556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2cd0f-c28d-4622-8988-4a6396b7e740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:16.401986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2cd0f-c28d-4622-8988-4a6396b7e740 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:16.430049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:16.497453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 573/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:57:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:57:16.503243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2e33e-d80c-4281-88f7-f1e71555b06c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:16.591518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:16.600703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2e33e-d80c-4281-88f7-f1e71555b06c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:16.633550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576cd0c-ab64-40a8-a504-5c1bee0a52a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:16.634162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd0c-ab64-40a8-a504-5c1bee0a52a9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:16.634440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd0c-ab64-40a8-a504-5c1bee0a52a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:16.634710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd0c-ab64-40a8-a504-5c1bee0a52a9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:16.641200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576cd0c-ab64-40a8-a504-5c1bee0a52a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:16.642635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576cd0c-ab64-40a8-a504-5c1bee0a52a9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5576cd0c-ab64-40a8-a504-5c1bee0a52a9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:57:16.654224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:57:16.655444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:57:16.656461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 592/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:57:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:57:16.661164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2cd0f-c28d-4622-8988-4a6396b7e740 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:16.757145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2cd0f-c28d-4622-8988-4a6396b7e740 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:16.785708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:16.791409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59db9a0-ed4a-40c5-a35d-a67cbad52d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:16.792013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59db9a0-ed4a-40c5-a35d-a67cbad52d9d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:16.792318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59db9a0-ed4a-40c5-a35d-a67cbad52d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:16.792590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59db9a0-ed4a-40c5-a35d-a67cbad52d9d None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:16.793221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59db9a0-ed4a-40c5-a35d-a67cbad52d9d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:16.793522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59db9a0-ed4a-40c5-a35d-a67cbad52d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:16.794880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59db9a0-ed4a-40c5-a35d-a67cbad52d9d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:16.801787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 574/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Aug 23 07:57:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:18.674894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac996295-3355-4c57-ab7c-3c2ccb7fcdc1 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:18.675468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac996295-3355-4c57-ab7c-3c2ccb7fcdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:18.675468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac996295-3355-4c57-ab7c-3c2ccb7fcdc1 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:18.676808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 593/1167] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:18.682016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61364a-434f-49de-b35d-6cee578e3b03 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:18.682391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61364a-434f-49de-b35d-6cee578e3b03 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:18.682578 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61364a-434f-49de-b35d-6cee578e3b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:18.700988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7e61364a-434f-49de-b35d-6cee578e3b03 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:18.702175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7e61364a-434f-49de-b35d-6cee578e3b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:18.767438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 575/1168] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:19.041369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac996295-3355-4c57-ab7c-3c2ccb7fcdc1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:19.140555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac996295-3355-4c57-ab7c-3c2ccb7fcdc1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:19.172625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4926c7c-549e-43d5-b4bb-962bc3ee4333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:19.173212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4926c7c-549e-43d5-b4bb-962bc3ee4333 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:19.173493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4926c7c-549e-43d5-b4bb-962bc3ee4333 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:19.173767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4926c7c-549e-43d5-b4bb-962bc3ee4333 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:19.174441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4926c7c-549e-43d5-b4bb-962bc3ee4333 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:19.175828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4926c7c-549e-43d5-b4bb-962bc3ee4333 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:19.250041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 594/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Tue Aug 23 07:57:19 2022] POST /identity/v3/domains => generated 267 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:57:21.159417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767764b9-45b1-4eb5-b67d-2702d1d87650 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:21.159927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767764b9-45b1-4eb5-b67d-2702d1d87650 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:21.159927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767764b9-45b1-4eb5-b67d-2702d1d87650 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:21.161230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 576/1170] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:21.166965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d65e7-9c47-4609-a8d1-bbd40505afcb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:21.167385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d65e7-9c47-4609-a8d1-bbd40505afcb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:21.167497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d65e7-9c47-4609-a8d1-bbd40505afcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:21.185678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e51d65e7-9c47-4609-a8d1-bbd40505afcb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:21.186902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e51d65e7-9c47-4609-a8d1-bbd40505afcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:21.189645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:21.252249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 595/1171] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:21.514306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767764b9-45b1-4eb5-b67d-2702d1d87650 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:21.617122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767764b9-45b1-4eb5-b67d-2702d1d87650 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:21.639483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:21.650200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67f3cf-11b6-462c-a93b-9ea062f40a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:21.650813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67f3cf-11b6-462c-a93b-9ea062f40a03 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:21.651129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67f3cf-11b6-462c-a93b-9ea062f40a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:21.651402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67f3cf-11b6-462c-a93b-9ea062f40a03 None admin] PATH_INFO: `/v3/domains/4c713a05eabf428e87b49ca3f80b3cf5` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:21.661497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67f3cf-11b6-462c-a93b-9ea062f40a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4c713a05eabf428e87b49ca3f80b3cf5)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:21.662961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67f3cf-11b6-462c-a93b-9ea062f40a03 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:21.665211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 577/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Tue Aug 23 07:57:21 2022] GET /identity/v3/domains/4c713a05eabf428e87b49ca3f80b3cf5 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:21.670870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d65e7-9c47-4609-a8d1-bbd40505afcb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:21.764611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d65e7-9c47-4609-a8d1-bbd40505afcb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:21.798056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c02c4-e545-409c-af6e-2991411c0251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:21.798655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c02c4-e545-409c-af6e-2991411c0251 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:21.798926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c02c4-e545-409c-af6e-2991411c0251 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:21.799217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c02c4-e545-409c-af6e-2991411c0251 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:21.799844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c02c4-e545-409c-af6e-2991411c0251 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:21.801256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c02c4-e545-409c-af6e-2991411c0251 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:21.811918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:21.895636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 596/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:57:21 2022] POST /identity/v3/projects => generated 361 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:57:23.823559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd959ed9-58ba-4d6a-8245-f093a97b571f None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:23.824142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd959ed9-58ba-4d6a-8245-f093a97b571f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:23.824142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd959ed9-58ba-4d6a-8245-f093a97b571f None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:23.825452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 578/1174] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:23.830807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639686-0f35-4090-988a-f82396426730 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:23.831254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639686-0f35-4090-988a-f82396426730 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:23.831415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e639686-0f35-4090-988a-f82396426730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:23.849302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7e639686-0f35-4090-988a-f82396426730 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:23.850474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7e639686-0f35-4090-988a-f82396426730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:23.915157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 597/1175] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:24.183257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd959ed9-58ba-4d6a-8245-f093a97b571f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:24.283076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd959ed9-58ba-4d6a-8245-f093a97b571f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:24.316488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1b6c9-656d-46e3-bcb0-ef6ab8b3837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:24.317105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1b6c9-656d-46e3-bcb0-ef6ab8b3837f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:24.317385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1b6c9-656d-46e3-bcb0-ef6ab8b3837f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:24.317658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1b6c9-656d-46e3-bcb0-ef6ab8b3837f None admin] PATH_INFO: `/v3/domains/4c713a05eabf428e87b49ca3f80b3cf5` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:24.318762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1b6c9-656d-46e3-bcb0-ef6ab8b3837f None admin] RBAC: Authorizing `identity:get_domain(domain_id=4c713a05eabf428e87b49ca3f80b3cf5)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:24.320227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1b6c9-656d-46e3-bcb0-ef6ab8b3837f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:24.321784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 579/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Tue Aug 23 07:57:24 2022] GET /identity/v3/domains/4c713a05eabf428e87b49ca3f80b3cf5 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:24.327745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e639686-0f35-4090-988a-f82396426730 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:24.424872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e639686-0f35-4090-988a-f82396426730 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:24.446353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:24.457650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6717e-efa4-487b-b47c-a28b4fc10cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:24.458240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6717e-efa4-487b-b47c-a28b4fc10cd9 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:24.459701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6717e-efa4-487b-b47c-a28b4fc10cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:24.459701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6717e-efa4-487b-b47c-a28b4fc10cd9 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:24.459701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6717e-efa4-487b-b47c-a28b4fc10cd9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:24.461442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6717e-efa4-487b-b47c-a28b4fc10cd9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:24.548349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 598/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:57:24 2022] POST /identity/v3/users => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:57:26.461471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf3d8b-e2ed-4be4-9449-755c87dbdc39 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:26.462070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf3d8b-e2ed-4be4-9449-755c87dbdc39 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:26.462070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf3d8b-e2ed-4be4-9449-755c87dbdc39 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:26.463476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 580/1178] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:26.468942 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6812297-17fc-433b-999d-68675307eed9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:26.469378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6812297-17fc-433b-999d-68675307eed9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:26.469500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6812297-17fc-433b-999d-68675307eed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:26.487448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c6812297-17fc-433b-999d-68675307eed9 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:26.488605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c6812297-17fc-433b-999d-68675307eed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:26.552566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 599/1179] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:57:26.817777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf3d8b-e2ed-4be4-9449-755c87dbdc39 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:26.872007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:26.919811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf3d8b-e2ed-4be4-9449-755c87dbdc39 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:26.952546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcce0f-d2ae-49ef-a072-94ede356a9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:26.953149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcce0f-d2ae-49ef-a072-94ede356a9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:26.953423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcce0f-d2ae-49ef-a072-94ede356a9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:26.953689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcce0f-d2ae-49ef-a072-94ede356a9d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:26.964545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcce0f-d2ae-49ef-a072-94ede356a9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-a0fcce0f-d2ae-49ef-a072-94ede356a9d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:57:26.969120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:57:26.970347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:57:26.971308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 581/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:57:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:57:26.976248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6812297-17fc-433b-999d-68675307eed9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:27.071404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6812297-17fc-433b-999d-68675307eed9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:27.080468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:27.104934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:27.105521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:27.105800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:27.106071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:27.106722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:27.107050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:27.108417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:27.117267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 600/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:57:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:27.122917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcce0f-d2ae-49ef-a072-94ede356a9d5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:27.218324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcce0f-d2ae-49ef-a072-94ede356a9d5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:27.231016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:27.251420 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd749-f7b9-4db1-bef0-e5758a25e054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:27.252025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd749-f7b9-4db1-bef0-e5758a25e054 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:27.252309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd749-f7b9-4db1-bef0-e5758a25e054 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:27.252583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd749-f7b9-4db1-bef0-e5758a25e054 None admin] PATH_INFO: `/v3/users/919bde84a2e44341ae695fd0c6739200` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:27.265728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cd749-f7b9-4db1-bef0-e5758a25e054 None admin] RBAC: Authorizing `identity:get_user(user_id=919bde84a2e44341ae695fd0c6739200)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:27.267277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cd749-f7b9-4db1-bef0-e5758a25e054 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:27.269057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 582/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:57:27 2022] GET /identity/v3/users/919bde84a2e44341ae695fd0c6739200 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:27.275311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:27.372342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da0c33-1ebb-4956-beb0-112c075d1117 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:27.404894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46909caa-5fea-4b9e-8987-a1e96a5b4c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:27.405488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46909caa-5fea-4b9e-8987-a1e96a5b4c0a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:27.405766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46909caa-5fea-4b9e-8987-a1e96a5b4c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:27.406037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46909caa-5fea-4b9e-8987-a1e96a5b4c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:27.443430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:27.471118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46909caa-5fea-4b9e-8987-a1e96a5b4c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:27.473015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46909caa-5fea-4b9e-8987-a1e96a5b4c0a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:27.566696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 601/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:57:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:57:27.572632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd749-f7b9-4db1-bef0-e5758a25e054 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:27.668487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd749-f7b9-4db1-bef0-e5758a25e054 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:27.701026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ced9e-8a2f-4c23-a184-bb2ba348575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:27.701634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ced9e-8a2f-4c23-a184-bb2ba348575c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:27.701907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ced9e-8a2f-4c23-a184-bb2ba348575c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:27.702174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ced9e-8a2f-4c23-a184-bb2ba348575c None admin] PATH_INFO: `/v3/projects/ad48ae5269cb4f8f836fcebd0b911cc0` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:27.703286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ced9e-8a2f-4c23-a184-bb2ba348575c None admin] RBAC: Authorizing `identity:get_project(project_id=ad48ae5269cb4f8f836fcebd0b911cc0)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:27.704689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ced9e-8a2f-4c23-a184-bb2ba348575c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:27.706248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 583/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:57:27 2022] GET /identity/v3/projects/ad48ae5269cb4f8f836fcebd0b911cc0 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:27.712670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46909caa-5fea-4b9e-8987-a1e96a5b4c0a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:27.779025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:27.808920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46909caa-5fea-4b9e-8987-a1e96a5b4c0a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:27.841677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b61ff-49c2-4d3c-b9db-621e15ba307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:27.842259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b61ff-49c2-4d3c-b9db-621e15ba307d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:27.842532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b61ff-49c2-4d3c-b9db-621e15ba307d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:27.842799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b61ff-49c2-4d3c-b9db-621e15ba307d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:27.843533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b61ff-49c2-4d3c-b9db-621e15ba307d None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=ad48ae5269cb4f8f836fcebd0b911cc0, user.id=919bde84a2e44341ae695fd0c6739200) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:27.843835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b61ff-49c2-4d3c-b9db-621e15ba307d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:27.845246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b61ff-49c2-4d3c-b9db-621e15ba307d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:27.850476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 602/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:57:27 2022] GET /identity/v3/role_assignments?user.id=919bde84a2e44341ae695fd0c6739200&scope.project.id=ad48ae5269cb4f8f836fcebd0b911cc0&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:29.712377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff24d0f-9e3b-42a3-a513-01066ca3146e None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:29.712942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff24d0f-9e3b-42a3-a513-01066ca3146e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:29.712942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff24d0f-9e3b-42a3-a513-01066ca3146e None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:29.714245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 584/1186] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:29.719636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50403f-8b12-4696-a15f-5f0f7097e22d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:29.720034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50403f-8b12-4696-a15f-5f0f7097e22d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:29.720183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50403f-8b12-4696-a15f-5f0f7097e22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:29.737824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cc50403f-8b12-4696-a15f-5f0f7097e22d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:29.739029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cc50403f-8b12-4696-a15f-5f0f7097e22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:29.804406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 603/1187] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:30.078378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff24d0f-9e3b-42a3-a513-01066ca3146e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:30.119146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:30.176706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff24d0f-9e3b-42a3-a513-01066ca3146e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:30.209555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a92c-0b0a-4c9b-9fbc-91b4be0d4055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:30.210191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a92c-0b0a-4c9b-9fbc-91b4be0d4055 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:30.210467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a92c-0b0a-4c9b-9fbc-91b4be0d4055 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:30.210738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a92c-0b0a-4c9b-9fbc-91b4be0d4055 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:30.220428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366a92c-0b0a-4c9b-9fbc-91b4be0d4055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b366a92c-0b0a-4c9b-9fbc-91b4be0d4055 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:57:30.225013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:57:30.226601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:57:30.227683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 585/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:57:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:57:30.232706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50403f-8b12-4696-a15f-5f0f7097e22d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:30.292357 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:30.329170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50403f-8b12-4696-a15f-5f0f7097e22d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:30.362048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:30.362656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:30.362908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:30.363211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:30.363860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:30.364166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:30.365527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:30.373258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 604/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:30.378879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a92c-0b0a-4c9b-9fbc-91b4be0d4055 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:30.473540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:30.477531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a92c-0b0a-4c9b-9fbc-91b4be0d4055 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:30.510696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741e4f9-f82b-4378-8799-447f998c51d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:30.511321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741e4f9-f82b-4378-8799-447f998c51d9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:30.511597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741e4f9-f82b-4378-8799-447f998c51d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:30.511864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741e4f9-f82b-4378-8799-447f998c51d9 None admin] PATH_INFO: `/v3/users/919bde84a2e44341ae695fd0c6739200` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:30.512961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741e4f9-f82b-4378-8799-447f998c51d9 None admin] RBAC: Authorizing `identity:get_user(user_id=919bde84a2e44341ae695fd0c6739200)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:30.514381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741e4f9-f82b-4378-8799-447f998c51d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:30.516004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 586/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:57:30 2022] GET /identity/v3/users/919bde84a2e44341ae695fd0c6739200 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:30.521694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:30.618411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784db8-a141-47f0-90a0-012409599a90 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:30.642626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:30.650892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f3e8c-5000-4598-bc01-9a96f3ff2d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:30.651526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f3e8c-5000-4598-bc01-9a96f3ff2d11 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:30.651801 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f3e8c-5000-4598-bc01-9a96f3ff2d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:30.652069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f3e8c-5000-4598-bc01-9a96f3ff2d11 None admin] PATH_INFO: `/v3/projects/ad48ae5269cb4f8f836fcebd0b911cc0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:30.653156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f3e8c-5000-4598-bc01-9a96f3ff2d11 None admin] RBAC: Authorizing `identity:get_project(project_id=ad48ae5269cb4f8f836fcebd0b911cc0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:30.654564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f3e8c-5000-4598-bc01-9a96f3ff2d11 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:30.656216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 605/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:57:30 2022] GET /identity/v3/projects/ad48ae5269cb4f8f836fcebd0b911cc0 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:30.661919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741e4f9-f82b-4378-8799-447f998c51d9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:30.758292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741e4f9-f82b-4378-8799-447f998c51d9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:30.791303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd5925-d52b-444c-9632-cb68dd93a6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:30.791939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd5925-d52b-444c-9632-cb68dd93a6c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:30.792218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd5925-d52b-444c-9632-cb68dd93a6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:30.792489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd5925-d52b-444c-9632-cb68dd93a6c6 None admin] PATH_INFO: `/v3/projects/ad48ae5269cb4f8f836fcebd0b911cc0/users/919bde84a2e44341ae695fd0c6739200/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:30.794366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd5925-d52b-444c-9632-cb68dd93a6c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad48ae5269cb4f8f836fcebd0b911cc0, user_id=919bde84a2e44341ae695fd0c6739200, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:30.795821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd5925-d52b-444c-9632-cb68dd93a6c6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:30.885033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 587/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:57:30 2022] PUT /identity/v3/projects/ad48ae5269cb4f8f836fcebd0b911cc0/users/919bde84a2e44341ae695fd0c6739200/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:57:32.741815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc1288-d7db-4753-b31e-8cf4897be0e6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:32.742375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc1288-d7db-4753-b31e-8cf4897be0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:32.742375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc1288-d7db-4753-b31e-8cf4897be0e6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:32.743735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 606/1193] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:57:32.748816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe9bca-015b-4993-95f7-6f9cb0455bb0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:32.749195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe9bca-015b-4993-95f7-6f9cb0455bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:32.749355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe9bca-015b-4993-95f7-6f9cb0455bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:32.767697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ddfe9bca-015b-4993-95f7-6f9cb0455bb0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:57:32.768872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ddfe9bca-015b-4993-95f7-6f9cb0455bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:57:32.788551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:32.834851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 588/1194] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:57:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:57:33.102165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc1288-d7db-4753-b31e-8cf4897be0e6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:33.200697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc1288-d7db-4753-b31e-8cf4897be0e6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:33.233231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf61fb3-c233-4525-95d2-276ba3b910ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:33.233838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf61fb3-c233-4525-95d2-276ba3b910ca None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:33.234115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf61fb3-c233-4525-95d2-276ba3b910ca None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:33.234386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf61fb3-c233-4525-95d2-276ba3b910ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:33.244306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf61fb3-c233-4525-95d2-276ba3b910ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9bf61fb3-c233-4525-95d2-276ba3b910ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:57:33.248950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:57:33.250079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:57:33.251146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 607/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:57:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:57:33.255923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe9bca-015b-4993-95f7-6f9cb0455bb0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:33.351000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe9bca-015b-4993-95f7-6f9cb0455bb0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:33.383231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:33.383761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:33.384033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:33.384299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:33.384940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:33.385234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:33.386587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:33.394223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 589/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:57:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:33.400294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf61fb3-c233-4525-95d2-276ba3b910ca None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:33.454609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:33.497461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf61fb3-c233-4525-95d2-276ba3b910ca None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:33.532326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153d411-161d-45a1-be14-a034ee307dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:33.532939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153d411-161d-45a1-be14-a034ee307dcd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:33.533218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153d411-161d-45a1-be14-a034ee307dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:33.533489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153d411-161d-45a1-be14-a034ee307dcd None admin] PATH_INFO: `/v3/users/919bde84a2e44341ae695fd0c6739200` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:33.534598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153d411-161d-45a1-be14-a034ee307dcd None admin] RBAC: Authorizing `identity:get_user(user_id=919bde84a2e44341ae695fd0c6739200)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:33.536063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153d411-161d-45a1-be14-a034ee307dcd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:33.537628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 608/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Aug 23 07:57:33 2022] GET /identity/v3/users/919bde84a2e44341ae695fd0c6739200 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:33.544251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:33.563222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:33.640986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437750b4-f060-4396-af43-ef6c3f82eddf None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:33.673797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea5393-b5a3-4819-ad10-4a35d598574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:33.674385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea5393-b5a3-4819-ad10-4a35d598574d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:33.674663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea5393-b5a3-4819-ad10-4a35d598574d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:33.674980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea5393-b5a3-4819-ad10-4a35d598574d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:33.739505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea5393-b5a3-4819-ad10-4a35d598574d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:33.740934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea5393-b5a3-4819-ad10-4a35d598574d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:33.759194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:33.837229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 590/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:57:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 07:57:33.843092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153d411-161d-45a1-be14-a034ee307dcd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:33.938573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153d411-161d-45a1-be14-a034ee307dcd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:33.960794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:33.971904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007094d-59b3-413b-9ff3-9f0639f5afd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:33.972518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007094d-59b3-413b-9ff3-9f0639f5afd3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:33.972800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007094d-59b3-413b-9ff3-9f0639f5afd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:33.973071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007094d-59b3-413b-9ff3-9f0639f5afd3 None admin] PATH_INFO: `/v3/projects/ad48ae5269cb4f8f836fcebd0b911cc0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:33.974173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007094d-59b3-413b-9ff3-9f0639f5afd3 None admin] RBAC: Authorizing `identity:get_project(project_id=ad48ae5269cb4f8f836fcebd0b911cc0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:33.976157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007094d-59b3-413b-9ff3-9f0639f5afd3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:33.977863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 609/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Aug 23 07:57:33 2022] GET /identity/v3/projects/ad48ae5269cb4f8f836fcebd0b911cc0 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:33.983986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea5393-b5a3-4819-ad10-4a35d598574d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:57:34.079375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea5393-b5a3-4819-ad10-4a35d598574d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:57:34.096171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:34.112777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b71a2b-fcc1-4428-93e3-f087006a9022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:57:34.113357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71a2b-fcc1-4428-93e3-f087006a9022 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:57:34.113635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71a2b-fcc1-4428-93e3-f087006a9022 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:57:34.113904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71a2b-fcc1-4428-93e3-f087006a9022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:57:34.114617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b71a2b-fcc1-4428-93e3-f087006a9022 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=ad48ae5269cb4f8f836fcebd0b911cc0, user.id=919bde84a2e44341ae695fd0c6739200) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:57:34.114919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b71a2b-fcc1-4428-93e3-f087006a9022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:57:34.116382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b71a2b-fcc1-4428-93e3-f087006a9022 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:57:34.121851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 591/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:57:33 2022] GET /identity/v3/role_assignments?user.id=919bde84a2e44341ae695fd0c6739200&scope.project.id=ad48ae5269cb4f8f836fcebd0b911cc0&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:57:43.988486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:57:44.128942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=9 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:50.295693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaf290-1392-4640-999f-49401bafc7b6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:50.296291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaf290-1392-4640-999f-49401bafc7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:50.296291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaf290-1392-4640-999f-49401bafc7b6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:50.297584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 610/1201] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:58:50.303260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0823-b85f-49c9-b8a1-5d1bbebb241b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:50.311541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0823-b85f-49c9-b8a1-5d1bbebb241b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:50.311541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f0823-b85f-49c9-b8a1-5d1bbebb241b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:50.321290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e72f0823-b85f-49c9-b8a1-5d1bbebb241b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:58:50.322455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e72f0823-b85f-49c9-b8a1-5d1bbebb241b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:58:50.387567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 592/1202] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:58:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:58:50.657094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaf290-1392-4640-999f-49401bafc7b6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:50.757486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaf290-1392-4640-999f-49401bafc7b6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:50.791081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:50.791695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:50.791965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:50.792231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:50.802525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:50.803985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:58:50.818012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:50.819243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:58:50.820287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:58:50.820287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:50.820287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:58:50.820287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 611/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:50.825409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0823-b85f-49c9-b8a1-5d1bbebb241b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:50.923867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f0823-b85f-49c9-b8a1-5d1bbebb241b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:50.953156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:50.959718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df64449-1ecd-4c8c-bd64-13872d083dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:50.959718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df64449-1ecd-4c8c-bd64-13872d083dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:50.960064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df64449-1ecd-4c8c-bd64-13872d083dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:50.960064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df64449-1ecd-4c8c-bd64-13872d083dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:50.960657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df64449-1ecd-4c8c-bd64-13872d083dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:50.960957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df64449-1ecd-4c8c-bd64-13872d083dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:50.962350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df64449-1ecd-4c8c-bd64-13872d083dd4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:50.973721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 593/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:50.979669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:51.030875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:51.076351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ce6b2-e636-4d86-b1c4-d86bc18cb124 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:51.108756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf0e80-5210-42b5-a63c-7b5a75abe02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:51.109350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf0e80-5210-42b5-a63c-7b5a75abe02a None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:51.109628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf0e80-5210-42b5-a63c-7b5a75abe02a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:51.109901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf0e80-5210-42b5-a63c-7b5a75abe02a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:51.110566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf0e80-5210-42b5-a63c-7b5a75abe02a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:51.112053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf0e80-5210-42b5-a63c-7b5a75abe02a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:51.221440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 612/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Aug 23 07:58:50 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:58:53.119611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf05f8-bbaa-4ce8-8597-992f2666e5a6 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:53.120115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf05f8-bbaa-4ce8-8597-992f2666e5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:53.120115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf05f8-bbaa-4ce8-8597-992f2666e5a6 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:53.121438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 594/1206] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:58:53.126624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd8e04-a302-4e64-bab3-baafce88d466 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:53.127064 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd8e04-a302-4e64-bab3-baafce88d466 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:53.131840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd8e04-a302-4e64-bab3-baafce88d466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:53.145965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-15bd8e04-a302-4e64-bab3-baafce88d466 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:58:53.147186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-15bd8e04-a302-4e64-bab3-baafce88d466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:58:53.214226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 613/1207] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:58:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:58:53.480917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf05f8-bbaa-4ce8-8597-992f2666e5a6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:53.581622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf05f8-bbaa-4ce8-8597-992f2666e5a6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:53.614405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14750825-08c0-4cda-adb8-eb22c4536264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:53.615042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750825-08c0-4cda-adb8-eb22c4536264 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:53.615319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750825-08c0-4cda-adb8-eb22c4536264 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:53.615589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750825-08c0-4cda-adb8-eb22c4536264 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:53.625850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14750825-08c0-4cda-adb8-eb22c4536264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-14750825-08c0-4cda-adb8-eb22c4536264 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:58:53.631026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:53.632155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:58:53.633011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 595/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:58:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:53.638117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd8e04-a302-4e64-bab3-baafce88d466 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:53.676402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:53.733376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd8e04-a302-4e64-bab3-baafce88d466 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:53.766471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:53.767122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:53.767380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:53.767646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:53.768302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:53.768600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:53.769985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:53.777621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 614/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:58:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:53.783455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14750825-08c0-4cda-adb8-eb22c4536264 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:53.834105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:53.883328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14750825-08c0-4cda-adb8-eb22c4536264 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:53.915954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:53.916571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:53.916849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:53.917150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:53.927934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:53.934300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:58:53.943113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:53.944199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:58:53.945350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:58:53.945350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:53.945350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:58:53.945350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 596/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:53.950429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:54.047766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:54.050651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161b232-33a3-4294-8529-e4212e47c8b0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:54.085993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:54.086580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:54.086853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:54.087147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:54.087774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:54.088069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:54.089414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:54.100850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 615/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:54.106673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:54.204740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c803412-53a9-4c46-a7b9-90e673584a30 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:54.206385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:54.238827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:54.239473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:54.239743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:54.240008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:54.247160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:54.248569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:58:54.259204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:54.260236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:58:54.261190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 597/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:58:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:54.266240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:54.362914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078ef69-e131-4913-b3db-e89c98ed1cdd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:54.395154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:54.395745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:54.396020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:54.396289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:54.396997 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:54.397297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:54.398718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:54.415811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 616/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Aug 23 07:58:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:54.422190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:54.520340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405038d-1ed5-4ba0-ad1d-d278df09dd1a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:54.553141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f701f2a-279d-4618-b79c-403579dc5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:54.559958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f701f2a-279d-4618-b79c-403579dc5457 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:54.559958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f701f2a-279d-4618-b79c-403579dc5457 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:54.559958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f701f2a-279d-4618-b79c-403579dc5457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:54.584165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:54.621465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f701f2a-279d-4618-b79c-403579dc5457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:54.622852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f701f2a-279d-4618-b79c-403579dc5457 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:54.720051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 598/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:58:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:58:54.726135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:54.744604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:54.823696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c6030-9632-4841-bc91-0130b6df88d6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:54.857372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:54.857989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:54.858265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:54.858531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:54.868388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:54.873323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:58:54.884449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:54.885670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:58:54.886781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:58:54.886781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:54.886781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:58:54.890247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 617/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:54.891465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f701f2a-279d-4618-b79c-403579dc5457 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:54.952150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:54.990478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f701f2a-279d-4618-b79c-403579dc5457 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:55.025222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:55.025815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:55.026095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:55.026368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:55.027063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:55.027370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:55.028746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:55.039704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 599/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:55.045371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:55.114113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:55.144817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531f00c-2c0b-4586-a012-d203bd2d4ceb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:55.178546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:55.179196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:55.179477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:55.179780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:55.186278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:55.187746 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:58:55.199464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:55.200559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:58:55.201618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 618/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:55.206641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:55.304887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e2f9a-3686-4fa6-a756-5792b2723c3f None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:55.326922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:55.337980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281f114-0f62-4df1-8580-25be5d7ad482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:55.338574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281f114-0f62-4df1-8580-25be5d7ad482 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:55.338850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281f114-0f62-4df1-8580-25be5d7ad482 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:55.339153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281f114-0f62-4df1-8580-25be5d7ad482 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:55.339828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281f114-0f62-4df1-8580-25be5d7ad482 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:55.340130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281f114-0f62-4df1-8580-25be5d7ad482 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:55.341544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281f114-0f62-4df1-8580-25be5d7ad482 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:55.353742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 600/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:55.360069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:55.460581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7bca-014e-4bdc-8a50-d8ed45f82116 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:55.488862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:55.494766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd910e4c-ad18-4825-ba2e-19eac0f3e584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:55.497704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd910e4c-ad18-4825-ba2e-19eac0f3e584 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:55.497704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd910e4c-ad18-4825-ba2e-19eac0f3e584 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:55.497704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd910e4c-ad18-4825-ba2e-19eac0f3e584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:55.497704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd910e4c-ad18-4825-ba2e-19eac0f3e584 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=40df52dd29044cddbd521b59dd30b6a0) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:55.497704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd910e4c-ad18-4825-ba2e-19eac0f3e584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:55.498372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd910e4c-ad18-4825-ba2e-19eac0f3e584 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:55.503714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 619/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:58:55 2022] GET /identity/v3/role_assignments?user.id=40df52dd29044cddbd521b59dd30b6a0&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:57.380905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5200c-078b-4e8f-b779-aea30c142758 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:57.381549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5200c-078b-4e8f-b779-aea30c142758 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:57.381549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5200c-078b-4e8f-b779-aea30c142758 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:57.382796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 601/1220] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:58:57.387868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0c3c3-3b8c-45a3-8bdc-06a4bae27513 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:57.388263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0c3c3-3b8c-45a3-8bdc-06a4bae27513 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:57.388492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0c3c3-3b8c-45a3-8bdc-06a4bae27513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:57.406657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e1d0c3c3-3b8c-45a3-8bdc-06a4bae27513 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:58:57.407871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e1d0c3c3-3b8c-45a3-8bdc-06a4bae27513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:58:57.474195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 620/1221] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:58:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:58:57.740952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5200c-078b-4e8f-b779-aea30c142758 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:57.843298 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5200c-078b-4e8f-b779-aea30c142758 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:57.876256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f659cf-dbee-4f15-9d5f-f221f3f11c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:57.876935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f659cf-dbee-4f15-9d5f-f221f3f11c99 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:57.877239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f659cf-dbee-4f15-9d5f-f221f3f11c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:57.877513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f659cf-dbee-4f15-9d5f-f221f3f11c99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:57.892663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f659cf-dbee-4f15-9d5f-f221f3f11c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d5f659cf-dbee-4f15-9d5f-f221f3f11c99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:58:57.898035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:57.903404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:58:57.904496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 602/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:58:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:57.905377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0c3c3-3b8c-45a3-8bdc-06a4bae27513 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:58.002540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0c3c3-3b8c-45a3-8bdc-06a4bae27513 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:58.026653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:58.036974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:58.037574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:58.037851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:58.038178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:58.038835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:58.039192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:58.040657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:58.048355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 621/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:58:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:58.054787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f659cf-dbee-4f15-9d5f-f221f3f11c99 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:58.100836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:58.155426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f659cf-dbee-4f15-9d5f-f221f3f11c99 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:58.188374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:58.188993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:58.189272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:58.189543 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:58.200124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:58.201553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:58:58.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:58.218530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:58:58.219711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:58:58.219711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:58.219711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:58:58.219711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 603/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:58.223352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:58.325834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e114f-b314-4d91-ac78-fc180b28957b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:58.357365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:58.357957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:58.358348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:58.358620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:58.359280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:58.359575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:58.360927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:58.372775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 622/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:58.378599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:58.469332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:58.476312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b35967-c14c-45ab-8a13-6c0e090004d9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:58.509243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:58.514297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:58.514297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:58.514297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:58.518519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:58.520610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:58:58.531358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:58.532548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:58:58.533558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 604/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:58:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:58.538350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:58.588574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:58.638953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6e3f-5379-4173-9424-0cc7514941b7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:58.671177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:58.671766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:58.672037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:58.672301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:58.673006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:58.673302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:58.674707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:58.692353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 623/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Aug 23 07:58:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:58.698149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:58.797314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd355f-e215-416d-aa44-bca8bba2a69b None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:58.832806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:58.833424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:58.833719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:58.833987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:58.845057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:58.846421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:58:58.860534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:58.864347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:58:58.865452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:58:58.865452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:58:58.865452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:58:58.865452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 605/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:58.867671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:58.967903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8fc0-8682-4c5d-ad61-24af64933f1b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:58.970196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:59.000959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:59.001549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:59.001826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:59.002098 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:59.002730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:59.003063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:59.004454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:59.015141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 624/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:59.021429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:59.030755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:59.119347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c24c1-4084-4576-8774-bb31e499e991 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:59.151929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:59.152544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:59.152823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:59.153092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:59.159653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:59.161678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:58:59.172930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:58:59.173979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:58:59.179318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 606/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:58:59.181250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:59.280577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b01cd-0be0-4b9c-9005-cd70df7dc357 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:59.313653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b5f03-3550-4b68-a5d9-a53cf6358f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:59.314242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b5f03-3550-4b68-a5d9-a53cf6358f5d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:59.314513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b5f03-3550-4b68-a5d9-a53cf6358f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:59.314779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b5f03-3550-4b68-a5d9-a53cf6358f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:59.315480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b5f03-3550-4b68-a5d9-a53cf6358f5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:58:59.315776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b5f03-3550-4b68-a5d9-a53cf6358f5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:59.317760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b5f03-3550-4b68-a5d9-a53cf6358f5d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:59.329505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 625/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:58:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:58:59.336139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:58:59.401081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:58:59.434778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc3b4c-fec5-4ef9-8fce-3aa1900f16d9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:58:59.468992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf466d-c692-453e-b7b3-e89e2dc06936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:58:59.469630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf466d-c692-453e-b7b3-e89e2dc06936 None admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:58:59.470146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf466d-c692-453e-b7b3-e89e2dc06936 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:58:59.470146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf466d-c692-453e-b7b3-e89e2dc06936 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/40df52dd29044cddbd521b59dd30b6a0/roles/43c2615f91944450952391b6a4dc693c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:58:59.484426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf466d-c692-453e-b7b3-e89e2dc06936 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=40df52dd29044cddbd521b59dd30b6a0, role_id=43c2615f91944450952391b6a4dc693c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:58:59.485859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf466d-c692-453e-b7b3-e89e2dc06936 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:58:59.563806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 607/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:58:59 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/40df52dd29044cddbd521b59dd30b6a0/roles/43c2615f91944450952391b6a4dc693c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:59:01.453368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a406a-d076-4e0c-9ffd-bb6e34cfe11b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:01.453943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a406a-d076-4e0c-9ffd-bb6e34cfe11b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:01.453943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a406a-d076-4e0c-9ffd-bb6e34cfe11b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:01.455268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 626/1233] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 07:59:01.461055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580794bd-0438-48b0-bb73-76a1e17364f8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:01.461429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580794bd-0438-48b0-bb73-76a1e17364f8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:01.466588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580794bd-0438-48b0-bb73-76a1e17364f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:01.479752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-580794bd-0438-48b0-bb73-76a1e17364f8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:59:01.480939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-580794bd-0438-48b0-bb73-76a1e17364f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:59:01.546830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 608/1234] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:59:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:59:01.816339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a406a-d076-4e0c-9ffd-bb6e34cfe11b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:01.842342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:01.916302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a406a-d076-4e0c-9ffd-bb6e34cfe11b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:01.950763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71388b20-944d-47b1-bd02-2745f2abc9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:01.951408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71388b20-944d-47b1-bd02-2745f2abc9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:01.951685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71388b20-944d-47b1-bd02-2745f2abc9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:01.951955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71388b20-944d-47b1-bd02-2745f2abc9e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:01.962662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71388b20-944d-47b1-bd02-2745f2abc9e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-71388b20-944d-47b1-bd02-2745f2abc9e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:59:01.967445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:01.970576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:01.971631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 627/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Aug 23 07:59:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:01.975310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580794bd-0438-48b0-bb73-76a1e17364f8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:02.020603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:02.072181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580794bd-0438-48b0-bb73-76a1e17364f8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:02.106301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:02.106893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:02.107207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:02.107479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:02.108134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:02.108458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:02.109835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:02.118147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 609/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Aug 23 07:59:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:02.123954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71388b20-944d-47b1-bd02-2745f2abc9e5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:02.208255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:02.223615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71388b20-944d-47b1-bd02-2745f2abc9e5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:02.256583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:02.257190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:02.257462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:02.257725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:02.267533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:02.268952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:59:02.285219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:02.286364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:59:02.287552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:59:02.287552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:02.287552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:02.287552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 628/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:02.292352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:02.399960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee9f17-c9c8-4e57-9e0d-baf8b070abd1 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:02.408658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:02.432954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:02.433549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:02.433827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:02.434097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:02.434728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:02.435061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:02.436423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:02.448238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 610/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:02.454153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:02.553762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0b131-696b-4a42-8436-c89ae62c3949 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:02.586822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:02.587461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:02.587739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:02.588007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:02.590613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:02.595457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:02.596881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:59:02.607669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:02.608843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:02.609833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 629/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:02.615243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:02.711764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652133b6-4852-4da5-8508-dbb2eaeb5509 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:02.744402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:02.744997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:02.745272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:02.745541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:02.746247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:02.746546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:02.748122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:02.765981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 611/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Aug 23 07:59:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:02.772338 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:02.870922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4605718-0267-41bc-acaf-37ba85522ebc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:02.903765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815554a8-e6dd-4b53-938b-4f464cf5e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:02.904362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815554a8-e6dd-4b53-938b-4f464cf5e31d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:02.904639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815554a8-e6dd-4b53-938b-4f464cf5e31d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:02.904908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815554a8-e6dd-4b53-938b-4f464cf5e31d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:02.964342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:02.970580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815554a8-e6dd-4b53-938b-4f464cf5e31d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:02.972059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815554a8-e6dd-4b53-938b-4f464cf5e31d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:03.068849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 630/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:59:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:59:03.075036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:03.102160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:03.171411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311a85b-1c44-4540-ba2a-33e7d8cdaed9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:03.204950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:03.206803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:03.206803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:03.206803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:03.216479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:03.217866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:59:03.234329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:03.236641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:59:03.238341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:59:03.238341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:03.238341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:03.238341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 612/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:03.241742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815554a8-e6dd-4b53-938b-4f464cf5e31d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:03.331104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:03.337184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815554a8-e6dd-4b53-938b-4f464cf5e31d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:03.369966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:03.370568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:03.370840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:03.371144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:03.371774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:03.372073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:03.373425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:03.384173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 631/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:03.389932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:03.473320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:03.494062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb36bd8-d7b0-47d3-9be7-01df82bdef1d None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:03.526949 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:03.527593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:03.527874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:03.528145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:03.534593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:03.536064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:59:03.546906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:03.547996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:03.549148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 613/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:59:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:03.554563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:03.653009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154960c-f9ab-46a3-9a2f-26e1357b9d90 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:03.686225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa6630-fc49-406f-a3e4-95ee0eb06808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:03.686819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6630-fc49-406f-a3e4-95ee0eb06808 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:03.687125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6630-fc49-406f-a3e4-95ee0eb06808 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:03.687394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6630-fc49-406f-a3e4-95ee0eb06808 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:03.688106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6630-fc49-406f-a3e4-95ee0eb06808 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:03.688404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6630-fc49-406f-a3e4-95ee0eb06808 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:03.689815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6630-fc49-406f-a3e4-95ee0eb06808 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:03.701674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 632/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:59:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:03.707714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:03.807949 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34153b2f-c3c2-47a5-aeb8-1a047567658c None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:03.841252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a3ca2-4956-467a-827f-7177aecb9199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:03.841852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a3ca2-4956-467a-827f-7177aecb9199 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:03.842130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a3ca2-4956-467a-827f-7177aecb9199 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:03.842403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a3ca2-4956-467a-827f-7177aecb9199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:03.843182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a3ca2-4956-467a-827f-7177aecb9199 None admin] RBAC: Adding query filter params (role.id=43c2615f91944450952391b6a4dc693c, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=40df52dd29044cddbd521b59dd30b6a0) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:03.843528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a3ca2-4956-467a-827f-7177aecb9199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:03.844953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a3ca2-4956-467a-827f-7177aecb9199 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:03.851162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 614/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:59:03 2022] GET /identity/v3/role_assignments?user.id=40df52dd29044cddbd521b59dd30b6a0&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=43c2615f91944450952391b6a4dc693c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:05.802144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb650b8-a76c-4b8a-8d61-e90c6822eedd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:05.802579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb650b8-a76c-4b8a-8d61-e90c6822eedd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:05.802679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb650b8-a76c-4b8a-8d61-e90c6822eedd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:05.804035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 633/1247] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:59:05.809239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf01d-78dd-47bb-8a30-4f836f07fd30 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:05.809661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf01d-78dd-47bb-8a30-4f836f07fd30 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:05.809813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf01d-78dd-47bb-8a30-4f836f07fd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:05.816146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:05.828863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d90cf01d-78dd-47bb-8a30-4f836f07fd30 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:59:05.830054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d90cf01d-78dd-47bb-8a30-4f836f07fd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:59:05.894891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 615/1248] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:59:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:59:06.165393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb650b8-a76c-4b8a-8d61-e90c6822eedd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:06.183457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:06.265548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb650b8-a76c-4b8a-8d61-e90c6822eedd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:06.298024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de810536-2680-410b-989c-6cf33a20f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:06.298624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de810536-2680-410b-989c-6cf33a20f043 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:06.298903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de810536-2680-410b-989c-6cf33a20f043 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:06.299203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de810536-2680-410b-989c-6cf33a20f043 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:06.309531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de810536-2680-410b-989c-6cf33a20f043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-de810536-2680-410b-989c-6cf33a20f043 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:59:06.314163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:06.315331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:06.317168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 634/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:59:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:06.321978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cf01d-78dd-47bb-8a30-4f836f07fd30 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:06.420257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cf01d-78dd-47bb-8a30-4f836f07fd30 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:06.449832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:06.454608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:06.455250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:06.455530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:06.455802 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:06.456481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:06.456781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:06.458204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:06.466162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 616/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:59:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 07:59:06.472001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de810536-2680-410b-989c-6cf33a20f043 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:06.547588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:06.570888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de810536-2680-410b-989c-6cf33a20f043 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:06.603676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:06.604298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:06.604575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:06.604844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:06.615720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:06.617161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:59:06.631080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:06.633537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:59:06.634646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:59:06.634646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:06.634646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:06.634646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 635/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:06.638291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:06.738893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200fd8-dbdd-4141-9280-1c6d8cc15ab4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:06.772040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:06.772636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:06.772910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:06.773176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:06.773803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:06.774099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:06.775527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:06.787315 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 617/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:06.792929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:06.889389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3530f-80e7-45d0-978c-f5d36465786d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:06.914606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:06.922163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:06.922770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:06.923076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:06.929193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:06.930436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:06.931889 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:59:06.942963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:06.944072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:06.945166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 636/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:06.949980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:07.005581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:07.048781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedecc1-f5fe-4e62-b96f-30e76a98cd93 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:07.082632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:07.083305 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:07.083587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:07.083861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:07.084578 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:07.084884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:07.086362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:07.103460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 618/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Aug 23 07:59:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:07.109346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:07.207185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd7a3b-3b77-4b76-b433-644d36c85cd8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:07.240476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671192-9552-48be-b381-ea2b5db6e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:07.241067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671192-9552-48be-b381-ea2b5db6e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:07.241346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671192-9552-48be-b381-ea2b5db6e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:07.241636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671192-9552-48be-b381-ea2b5db6e2f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:07.296196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:07.308526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25671192-9552-48be-b381-ea2b5db6e2f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:07.309947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25671192-9552-48be-b381-ea2b5db6e2f5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:07.408336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 637/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:59:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:59:07.415289 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:07.513302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362be6d-2b34-46b8-8891-7b1c640de21a None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:07.527694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:07.547634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:07.548189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:07.548466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:07.548734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:07.558308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:07.559811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:59:07.575182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:07.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:59:07.577422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:59:07.577422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:07.577422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:07.577422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 619/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:07.582685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671192-9552-48be-b381-ea2b5db6e2f5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:07.673514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:07.684853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671192-9552-48be-b381-ea2b5db6e2f5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:07.718099 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:07.718694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:07.718995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:07.719268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:07.719894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:07.720191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:07.721543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:07.732270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 638/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:07.737916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:07.836605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ced8-f682-42fd-aee9-900bc3ab68e0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:07.868712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:07.869318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:07.869593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:07.869859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:07.876323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:07.877745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:59:07.889217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:07.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:07.891258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 620/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:59:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:07.896379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:08.001740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051a1cc-ae2f-4612-b24d-14acb1afe594 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:08.036477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d451d-cae0-404d-bdb8-29a509da2fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:08.037104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d451d-cae0-404d-bdb8-29a509da2fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:08.037381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d451d-cae0-404d-bdb8-29a509da2fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:08.037641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d451d-cae0-404d-bdb8-29a509da2fb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:08.038322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d451d-cae0-404d-bdb8-29a509da2fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:08.038619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d451d-cae0-404d-bdb8-29a509da2fb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:08.040081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d451d-cae0-404d-bdb8-29a509da2fb6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:08.052326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 639/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:59:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:08.058523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:08.086394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:08.162377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e83cf-aa1a-47ee-bf18-0e5ec078c3d9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:08.194569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca913e-bf05-4835-be6d-8e46d92392a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:08.195189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca913e-bf05-4835-be6d-8e46d92392a6 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:08.195466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca913e-bf05-4835-be6d-8e46d92392a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:08.195733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca913e-bf05-4835-be6d-8e46d92392a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:08.196445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca913e-bf05-4835-be6d-8e46d92392a6 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=40df52dd29044cddbd521b59dd30b6a0) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:08.196747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca913e-bf05-4835-be6d-8e46d92392a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:08.198157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca913e-bf05-4835-be6d-8e46d92392a6 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:08.203685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 621/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:59:08 2022] GET /identity/v3/role_assignments?user.id=40df52dd29044cddbd521b59dd30b6a0&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:10.067760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2f43d-a8dd-47ba-9a11-7e05f47fe61c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:10.068261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2f43d-a8dd-47ba-9a11-7e05f47fe61c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:10.068261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2f43d-a8dd-47ba-9a11-7e05f47fe61c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:10.069564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 640/1261] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:59:10.075351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab332770-1b31-4ec2-8b86-09b0939ff398 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:10.075748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab332770-1b31-4ec2-8b86-09b0939ff398 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:10.075893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab332770-1b31-4ec2-8b86-09b0939ff398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:10.094261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ab332770-1b31-4ec2-8b86-09b0939ff398 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:59:10.095481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ab332770-1b31-4ec2-8b86-09b0939ff398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:59:10.162618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 622/1262] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:59:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:59:10.428548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2f43d-a8dd-47ba-9a11-7e05f47fe61c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:10.439397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:10.532436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2f43d-a8dd-47ba-9a11-7e05f47fe61c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:10.564804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d49f9-6c1e-4d95-89e6-f7abbd9d48cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:10.565415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d49f9-6c1e-4d95-89e6-f7abbd9d48cb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:10.566214 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d49f9-6c1e-4d95-89e6-f7abbd9d48cb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:10.566475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d49f9-6c1e-4d95-89e6-f7abbd9d48cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:10.576389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d49f9-6c1e-4d95-89e6-f7abbd9d48cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-794d49f9-6c1e-4d95-89e6-f7abbd9d48cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:59:10.581044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:10.582758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:10.583786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 641/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:10.589033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab332770-1b31-4ec2-8b86-09b0939ff398 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:10.606875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:10.687649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab332770-1b31-4ec2-8b86-09b0939ff398 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:10.722305 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:10.722906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:10.723221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:10.723493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:10.724155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:10.724457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:10.725822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:10.734141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 623/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:10.739900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d49f9-6c1e-4d95-89e6-f7abbd9d48cb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:10.808060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:10.840442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d49f9-6c1e-4d95-89e6-f7abbd9d48cb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:10.873342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:10.873960 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:10.874250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:10.874504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:10.885121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:10.886568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:59:10.900770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:10.903875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:59:10.905024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:59:10.905024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:10.905024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:10.905024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 642/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:10.907848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:10.990222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:11.008941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ff528-ac14-4595-9c32-2b9bed5e5d70 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:11.041115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:11.041717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:11.041997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:11.042270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:11.042918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:11.043239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:11.044602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:11.056646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 624/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:11.062341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:11.161219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0910a1b-0daf-4824-9505-aab405375933 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:11.180560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:11.194136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:11.194754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:11.195059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:11.195333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:11.202740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:11.204217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:59:11.214815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:11.220176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:11.221400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 643/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:11.222709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:11.320969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab0d9b-8e04-4022-bd47-30a345d38194 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:11.354135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:11.354689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:11.354986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:11.355254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:11.355965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:11.356263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:11.357698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:11.366595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:11.374618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 625/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Aug 23 07:59:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:11.380466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:11.479664 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e60000-9619-404c-a9d8-a5abe65c4550 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:11.513028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:11.513665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:11.513943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:11.515613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:11.524563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:11.525995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:59:11.540543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:11.545370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:59:11.546465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:59:11.546465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:11.546465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:11.546465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 644/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:11.547745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:11.646546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c817b-8b6d-40ae-a8a3-fff692afed63 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:11.680177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:11.680834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:11.681115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:11.681383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:11.682018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:11.682341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:11.683793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:11.694859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 626/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:11.700969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:11.733939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:11.805308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bcd8d-4f33-421e-be78-18c569baf9ac None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:11.839246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:11.839864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:11.840140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:11.840407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:11.847452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:11.848938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:59:11.859822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:11.860856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:11.863797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 645/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:59:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:11.867130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:11.924525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:11.965475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3112197-be4c-44ed-8d40-c8bb98e87cd5 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:11.997511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1684a1-533c-48e8-83de-6a29c2d8de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:11.997991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1684a1-533c-48e8-83de-6a29c2d8de61 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:11.998263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1684a1-533c-48e8-83de-6a29c2d8de61 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:11.998532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1684a1-533c-48e8-83de-6a29c2d8de61 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:11.999241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1684a1-533c-48e8-83de-6a29c2d8de61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:11.999556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1684a1-533c-48e8-83de-6a29c2d8de61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:12.000998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1684a1-533c-48e8-83de-6a29c2d8de61 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:12.012990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 627/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:59:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:12.019604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:12.108645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:12.116933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa442a-4f2d-4b39-959b-944ed498896c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:12.150303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e799bc2-d96a-4f93-8c6f-f02a297de136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:12.150966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799bc2-d96a-4f93-8c6f-f02a297de136 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:12.151239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799bc2-d96a-4f93-8c6f-f02a297de136 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:12.151506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e799bc2-d96a-4f93-8c6f-f02a297de136 None admin] PATH_INFO: `/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/40df52dd29044cddbd521b59dd30b6a0/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:12.153350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e799bc2-d96a-4f93-8c6f-f02a297de136 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1c63553f4ca4c1eacf87634168afe18, user_id=40df52dd29044cddbd521b59dd30b6a0, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:12.154755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e799bc2-d96a-4f93-8c6f-f02a297de136 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:12.251816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 646/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Aug 23 07:59:12 2022] PUT /identity/v3/projects/b1c63553f4ca4c1eacf87634168afe18/users/40df52dd29044cddbd521b59dd30b6a0/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 07:59:14.114817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148023-aa45-4901-a4d8-4db8867e054e None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:14.115353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148023-aa45-4901-a4d8-4db8867e054e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:14.115353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb148023-aa45-4901-a4d8-4db8867e054e None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:14.116679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 628/1274] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:59:14.121893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337c08-b037-4236-a7c2-4cb133e4d346 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:14.122274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337c08-b037-4236-a7c2-4cb133e4d346 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:14.122432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337c08-b037-4236-a7c2-4cb133e4d346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:14.141636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1d337c08-b037-4236-a7c2-4cb133e4d346 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:59:14.142832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1d337c08-b037-4236-a7c2-4cb133e4d346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:59:14.207262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 647/1275] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:59:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:59:14.469669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148023-aa45-4901-a4d8-4db8867e054e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:14.570434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb148023-aa45-4901-a4d8-4db8867e054e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:14.587365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:14.603434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddc651-b773-4e2b-9d0e-4f24578e86cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:14.604051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc651-b773-4e2b-9d0e-4f24578e86cf None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:14.604328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc651-b773-4e2b-9d0e-4f24578e86cf None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:14.604598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc651-b773-4e2b-9d0e-4f24578e86cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:14.614781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddc651-b773-4e2b-9d0e-4f24578e86cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-15ddc651-b773-4e2b-9d0e-4f24578e86cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 07:59:14.619405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:14.620405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:14.621263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 629/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Aug 23 07:59:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:14.626658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337c08-b037-4236-a7c2-4cb133e4d346 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:14.706647 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:14.723977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337c08-b037-4236-a7c2-4cb133e4d346 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:14.757355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:14.757965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:14.758245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:14.758517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:14.759203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:14.759504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:14.760882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:14.768892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 648/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Aug 23 07:59:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:14.774745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddc651-b773-4e2b-9d0e-4f24578e86cf None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:14.875055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddc651-b773-4e2b-9d0e-4f24578e86cf None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:14.914507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:14.915172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:14.915454 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:14.915725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:14.925874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:14.927354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:59:14.942300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:14.943469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:59:14.944827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:59:14.944827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:14.944827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:14.944827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 630/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:14.949738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:15.047358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3d629-8c95-4f52-b66c-630cd30aa7b2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:15.080107 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:15.085854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:15.085854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:15.085854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:15.085854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:15.085854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:15.085854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:15.095538 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 649/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:15.101130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:15.135448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:15.199608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67faeec-bdcb-43a8-bbc9-6e9faeef68d7 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:15.232659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:15.233274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:15.233556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:15.233825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:15.241150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:15.242590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 07:59:15.253818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:15.260279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:15.261434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 631/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:15.262179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:15.264739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:15.376882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ad0f7-d64a-4097-b6fa-e3a1f4040881 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:15.410024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:15.410620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:15.410897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:15.411198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:15.411911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:15.412211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:15.413636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:15.430519 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 650/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Aug 23 07:59:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:15.436392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:15.524214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:15.533750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90392e78-cb4e-4a2b-8f93-3eaebcd1ddc4 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:15.567468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce9ff4-2600-4ef5-877a-dce5a78671f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:15.568055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9ff4-2600-4ef5-877a-dce5a78671f2 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:15.568329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9ff4-2600-4ef5-877a-dce5a78671f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:15.568595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9ff4-2600-4ef5-877a-dce5a78671f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:15.633113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce9ff4-2600-4ef5-877a-dce5a78671f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:15.634464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce9ff4-2600-4ef5-877a-dce5a78671f2 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:15.718905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:15.729194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 632/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Aug 23 07:59:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 07:59:15.735191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:15.795747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:15.832888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de6f81-1488-46d3-93bd-73b85b53bcd5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:15.865724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:15.866333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:15.866607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:15.866871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:15.877616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:15.879116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 07:59:15.892547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:15.893703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 07:59:15.894632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 07:59:15.894632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 07:59:15.894632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:15.894632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 651/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 07:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:15.899370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce9ff4-2600-4ef5-877a-dce5a78671f2 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:16.001500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce9ff4-2600-4ef5-877a-dce5a78671f2 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:16.035733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:16.036325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:16.036597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:16.036910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:16.037532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:16.037827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:16.039255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:16.051001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 633/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Aug 23 07:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:16.056831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:16.157505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caa250-366e-4006-a32f-cc04e715e465 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:16.171631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:16.190881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:16.191528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:16.191806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:16.192074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:16.198881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:16.202398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 07:59:16.211794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:16.212860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Aug 23 07:59:16.214132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 652/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Aug 23 07:59:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:16.218827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:16.269151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:16.319434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8bdaf-0271-4f1a-a79e-5bd9341b24d9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:16.353125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fabe9-b6e8-42b7-b7a3-924e47806238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:16.353722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fabe9-b6e8-42b7-b7a3-924e47806238 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:16.354016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fabe9-b6e8-42b7-b7a3-924e47806238 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:16.354266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fabe9-b6e8-42b7-b7a3-924e47806238 None admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:16.354962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fabe9-b6e8-42b7-b7a3-924e47806238 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:16.355272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fabe9-b6e8-42b7-b7a3-924e47806238 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:16.356681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fabe9-b6e8-42b7-b7a3-924e47806238 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:16.368533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 634/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Aug 23 07:59:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:16.374721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:16.474632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1ee70-e859-4066-ac17-6bc8d03963b4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:16.507653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bfd25-f98d-436a-b38d-a60e2b8b40c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:16.508368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfd25-f98d-436a-b38d-a60e2b8b40c7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:16.508645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfd25-f98d-436a-b38d-a60e2b8b40c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:16.508913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfd25-f98d-436a-b38d-a60e2b8b40c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:16.509633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bfd25-f98d-436a-b38d-a60e2b8b40c7 None admin] RBAC: Adding query filter params (role.id=63edd667e7714fc08d3f32f6682c6369, scope.project.id=b1c63553f4ca4c1eacf87634168afe18, user.id=40df52dd29044cddbd521b59dd30b6a0) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:16.509935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bfd25-f98d-436a-b38d-a60e2b8b40c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:16.511390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bfd25-f98d-436a-b38d-a60e2b8b40c7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:16.517074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 653/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Aug 23 07:59:16 2022] GET /identity/v3/role_assignments?user.id=40df52dd29044cddbd521b59dd30b6a0&scope.project.id=b1c63553f4ca4c1eacf87634168afe18&role.id=63edd667e7714fc08d3f32f6682c6369 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:18.441384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bef32b-c784-4922-973b-3b8c8a5edb11 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:18.441934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bef32b-c784-4922-973b-3b8c8a5edb11 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:18.442030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bef32b-c784-4922-973b-3b8c8a5edb11 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:18.443354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 635/1288] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:59:18.448721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85464d3d-da98-4c9e-a563-5b025c350a02 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:18.449127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85464d3d-da98-4c9e-a563-5b025c350a02 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:18.449269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85464d3d-da98-4c9e-a563-5b025c350a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:18.467525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-85464d3d-da98-4c9e-a563-5b025c350a02 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:59:18.468754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-85464d3d-da98-4c9e-a563-5b025c350a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:59:18.497754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:18.533341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 654/1289] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:59:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 07:59:18.800570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bef32b-c784-4922-973b-3b8c8a5edb11 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:18.901065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bef32b-c784-4922-973b-3b8c8a5edb11 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:18.911575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:18.934024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:18.934642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:18.934918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:18.935227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:18.935801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:18.937212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:18.946390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:18.947642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:18.948794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 636/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:59:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:18.953945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85464d3d-da98-4c9e-a563-5b025c350a02 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:19.051999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85464d3d-da98-4c9e-a563-5b025c350a02 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:19.086330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950e43e-3f29-4fb2-a270-d536b4a3725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:19.086934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e43e-3f29-4fb2-a270-d536b4a3725b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:19.087239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e43e-3f29-4fb2-a270-d536b4a3725b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:19.087509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e43e-3f29-4fb2-a270-d536b4a3725b None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:19.088131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950e43e-3f29-4fb2-a270-d536b4a3725b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:19.088432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950e43e-3f29-4fb2-a270-d536b4a3725b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:19.089797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950e43e-3f29-4fb2-a270-d536b4a3725b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:19.094478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 655/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Tue Aug 23 07:59:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:19.100236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:19.199946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd88077-85b5-4084-a438-ba00d755c7a0 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:19.232595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3874b4-f027-4c8b-bf54-218e114c5119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:19.233195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3874b4-f027-4c8b-bf54-218e114c5119 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:19.233477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3874b4-f027-4c8b-bf54-218e114c5119 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:19.233750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3874b4-f027-4c8b-bf54-218e114c5119 None admin] PATH_INFO: `/v3/services` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:19.234376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3874b4-f027-4c8b-bf54-218e114c5119 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:19.234680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3874b4-f027-4c8b-bf54-218e114c5119 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:19.236120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3874b4-f027-4c8b-bf54-218e114c5119 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:19.241065 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 637/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Tue Aug 23 07:59:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:21.104327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a08dd-9a97-4ea1-b551-c8717073b392 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:21.104829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a08dd-9a97-4ea1-b551-c8717073b392 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:21.104829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a08dd-9a97-4ea1-b551-c8717073b392 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:21.106171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 656/1293] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:59:21.111316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bea1d-cb4d-45b5-8fad-b34c977cc4ad None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:21.111832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bea1d-cb4d-45b5-8fad-b34c977cc4ad None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:21.111832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bea1d-cb4d-45b5-8fad-b34c977cc4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:21.130189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ba8bea1d-cb4d-45b5-8fad-b34c977cc4ad None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:59:21.131395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ba8bea1d-cb4d-45b5-8fad-b34c977cc4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:59:21.160240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:21.195593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 638/1294] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:59:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:59:21.464883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a08dd-9a97-4ea1-b551-c8717073b392 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:21.496034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:21.563914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a08dd-9a97-4ea1-b551-c8717073b392 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:21.597992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4d497-0fc0-4410-ae1b-ffee56184da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:21.598584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d497-0fc0-4410-ae1b-ffee56184da7 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:21.598858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d497-0fc0-4410-ae1b-ffee56184da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:21.599150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d497-0fc0-4410-ae1b-ffee56184da7 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:21.599783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d497-0fc0-4410-ae1b-ffee56184da7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:21.601129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d497-0fc0-4410-ae1b-ffee56184da7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:21.718294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 657/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Aug 23 07:59:21 2022] POST /identity/v3/services => generated 246 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:59:23.617758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58895650-c9d9-4638-bbb4-19d24015ce19 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:23.618305 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58895650-c9d9-4638-bbb4-19d24015ce19 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:23.618305 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58895650-c9d9-4638-bbb4-19d24015ce19 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:23.619709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 639/1296] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:59:23.625817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f1dc-a68b-4b73-a4f9-fb681befe914 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:23.626352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f1dc-a68b-4b73-a4f9-fb681befe914 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:23.626352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f1dc-a68b-4b73-a4f9-fb681befe914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:23.647023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f7c0f1dc-a68b-4b73-a4f9-fb681befe914 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:59:23.648257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f7c0f1dc-a68b-4b73-a4f9-fb681befe914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:59:23.728794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 658/1297] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:59:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:59:23.999844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58895650-c9d9-4638-bbb4-19d24015ce19 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:24.102553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58895650-c9d9-4638-bbb4-19d24015ce19 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:24.137208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:24.137825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:24.138104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:24.138376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:24.139067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:24.140450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:24.149074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:24.150124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:24.151007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 640/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:59:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:24.156420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0f1dc-a68b-4b73-a4f9-fb681befe914 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:24.175623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:24.254683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0f1dc-a68b-4b73-a4f9-fb681befe914 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:24.287699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:24.288297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:24.288576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:24.288850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:24.289502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:24.289804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:24.291234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:24.298782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 659/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Tue Aug 23 07:59:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:24.303299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:24.335506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:24.403869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb03caf-8521-4dd4-8e1f-a21c62ea14a9 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:24.436941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c839cf-23ce-4dfc-9482-385cf0640f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:24.437533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c839cf-23ce-4dfc-9482-385cf0640f3d None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:24.437809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c839cf-23ce-4dfc-9482-385cf0640f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:24.438079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c839cf-23ce-4dfc-9482-385cf0640f3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:24.438763 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c839cf-23ce-4dfc-9482-385cf0640f3d None admin] RBAC: Adding query filter params (interface=public, service_id=f0f5bcf7640f4b30a0efcf1de81174fd, region_id=RegionOne) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:24.439098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c839cf-23ce-4dfc-9482-385cf0640f3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:24.441088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c839cf-23ce-4dfc-9482-385cf0640f3d None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:24.445887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 641/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Aug 23 07:59:24 2022] GET /identity/v3/endpoints?service_id=f0f5bcf7640f4b30a0efcf1de81174fd&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:24.451810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:24.533514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:24.551420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d0dd-7fb0-4a39-b1df-a1b2b272d1c1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:24.584624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422af17-321a-49a8-bbdc-68a0fd172ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:24.585217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422af17-321a-49a8-bbdc-68a0fd172ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:24.585492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422af17-321a-49a8-bbdc-68a0fd172ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:24.585763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422af17-321a-49a8-bbdc-68a0fd172ac8 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:24.586353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422af17-321a-49a8-bbdc-68a0fd172ac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:24.587731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422af17-321a-49a8-bbdc-68a0fd172ac8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:24.594610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 660/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Aug 23 07:59:24 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 07:59:26.484682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb96c3-5377-41ed-8986-b2085ae9810c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:26.485244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb96c3-5377-41ed-8986-b2085ae9810c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:26.485244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb96c3-5377-41ed-8986-b2085ae9810c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:26.486608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 642/1302] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 07:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 07:59:26.492556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da517802-4906-4a09-a84d-e12f53758037 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:26.493002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da517802-4906-4a09-a84d-e12f53758037 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:26.493104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da517802-4906-4a09-a84d-e12f53758037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:26.510595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-da517802-4906-4a09-a84d-e12f53758037 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 07:59:26.512619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-da517802-4906-4a09-a84d-e12f53758037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 07:59:26.576330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 661/1303] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 07:59:26 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 07:59:26.850908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb96c3-5377-41ed-8986-b2085ae9810c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:26.946103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:26.953100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb96c3-5377-41ed-8986-b2085ae9810c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:26.986083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:26.986700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:26.987049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:26.987324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:26.987896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:26.989287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 07:59:26.997978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 07:59:26.999042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application with Lock( Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self._enter() Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application return self.creator() Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application created_value = creator( Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: ERROR keystone.server.flask.application Aug 23 07:59:27.006104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 643/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Aug 23 07:59:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 07:59:27.007164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da517802-4906-4a09-a84d-e12f53758037 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:27.080146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:27.102213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da517802-4906-4a09-a84d-e12f53758037 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:27.134667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230898b-a23b-428e-bd6c-d329c30e623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:27.135294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230898b-a23b-428e-bd6c-d329c30e623c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:27.135574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230898b-a23b-428e-bd6c-d329c30e623c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:27.135846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230898b-a23b-428e-bd6c-d329c30e623c None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:27.136506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230898b-a23b-428e-bd6c-d329c30e623c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 07:59:27.136807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230898b-a23b-428e-bd6c-d329c30e623c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:27.138165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230898b-a23b-428e-bd6c-d329c30e623c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:27.143822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 662/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Tue Aug 23 07:59:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 07:59:27.149450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 07:59:27.246369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1345c50-7f9c-48a0-85ad-9eb8646d4481 None admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 07:59:27.284580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c8745-bb25-4f6d-a63d-ad686947f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 07:59:27.285169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c8745-bb25-4f6d-a63d-ad686947f2a4 None admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 07:59:27.285445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c8745-bb25-4f6d-a63d-ad686947f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 07:59:27.286033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c8745-bb25-4f6d-a63d-ad686947f2a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 07:59:27.286688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c8745-bb25-4f6d-a63d-ad686947f2a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 07:59:27.288164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c8745-bb25-4f6d-a63d-ad686947f2a4 None admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 07:59:27.389508 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 644/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 07:59:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 07:59:37.194532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=30 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 07:59:37.384767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:13.789673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa423-42c6-400d-b204-d9adc69e9086 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:13.790220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa423-42c6-400d-b204-d9adc69e9086 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:13.790304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa423-42c6-400d-b204-d9adc69e9086 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:13.792139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 663/1307] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:13.797218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4e50-4d55-4c20-8e8e-711bd1aec416 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:13.797657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4e50-4d55-4c20-8e8e-711bd1aec416 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:13.797882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4e50-4d55-4c20-8e8e-711bd1aec416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:13.887978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4afb4e50-4d55-4c20-8e8e-711bd1aec416 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:13.889293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4afb4e50-4d55-4c20-8e8e-711bd1aec416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:14.020367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 645/1308] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:14.315368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c06c3-8e4d-4471-8a9c-b7af3a6d89e9 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:14.315736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c06c3-8e4d-4471-8a9c-b7af3a6d89e9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:14.316469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c06c3-8e4d-4471-8a9c-b7af3a6d89e9 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:14.318022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 664/1309] 10.222.0.58 () {54 vars in 965 bytes} [Tue Aug 23 08:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:14.324170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbcb3b-b611-4531-9d6b-c5b533dbf3ab None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:14.324505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbcb3b-b611-4531-9d6b-c5b533dbf3ab None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:14.324792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbcb3b-b611-4531-9d6b-c5b533dbf3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:14.414445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a6fbcb3b-b611-4531-9d6b-c5b533dbf3ab None None] MFA Rules not processed for user `d99632164b3b4a1f95736a65da217db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:14.415754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a6fbcb3b-b611-4531-9d6b-c5b533dbf3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:14.439805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:14.504873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 646/1310] 10.222.0.58 () {58 vars in 1062 bytes} [Tue Aug 23 08:07:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:14.514159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c06c3-8e4d-4471-8a9c-b7af3a6d89e9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:14.515954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-935c06c3-8e4d-4471-8a9c-b7af3a6d89e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:14.643317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c06c3-8e4d-4471-8a9c-b7af3a6d89e9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:14.721888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e8f94-dfb0-4f4b-89b3-519262e191f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:14.722511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e8f94-dfb0-4f4b-89b3-519262e191f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:14.722785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e8f94-dfb0-4f4b-89b3-519262e191f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:14.723081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e8f94-dfb0-4f4b-89b3-519262e191f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:14.724675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ce6e8f94-dfb0-4f4b-89b3-519262e191f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:14.788470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:14.845575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e8f94-dfb0-4f4b-89b3-519262e191f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:14.847576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e8f94-dfb0-4f4b-89b3-519262e191f2 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:14.867498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 665/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:17.070797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ab1b9-13a2-419d-a9dc-0113c6f20138 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:17.071363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ab1b9-13a2-419d-a9dc-0113c6f20138 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:17.071363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ab1b9-13a2-419d-a9dc-0113c6f20138 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:17.073210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 647/1312] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:17.078924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641a9c3-109c-4a56-a341-37a78580623b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:17.079512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641a9c3-109c-4a56-a341-37a78580623b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:17.079512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641a9c3-109c-4a56-a341-37a78580623b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:17.176448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4641a9c3-109c-4a56-a341-37a78580623b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:17.177763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4641a9c3-109c-4a56-a341-37a78580623b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:17.279998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 666/1313] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:17.566580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ab1b9-13a2-419d-a9dc-0113c6f20138 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:17.568216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aa3ab1b9-13a2-419d-a9dc-0113c6f20138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:17.673413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ab1b9-13a2-419d-a9dc-0113c6f20138 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:17.746134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:17.748904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae6bfa-9864-464d-b36d-dd3b5b274141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:17.749515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6bfa-9864-464d-b36d-dd3b5b274141 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:17.749792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6bfa-9864-464d-b36d-dd3b5b274141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:17.750058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6bfa-9864-464d-b36d-dd3b5b274141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:17.751962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9eae6bfa-9864-464d-b36d-dd3b5b274141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:17.865534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae6bfa-9864-464d-b36d-dd3b5b274141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:17.867049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae6bfa-9864-464d-b36d-dd3b5b274141 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:17.887628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 648/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:20.295551 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecf494-b138-4ebe-9fc5-7aa5714a53dd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:20.296175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecf494-b138-4ebe-9fc5-7aa5714a53dd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:20.296175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecf494-b138-4ebe-9fc5-7aa5714a53dd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:20.297434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 667/1315] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:20.303260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6af236-745b-4026-88b8-af2208230bd7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:20.303656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6af236-745b-4026-88b8-af2208230bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:20.303817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6af236-745b-4026-88b8-af2208230bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:20.369762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4c6af236-745b-4026-88b8-af2208230bd7 None None] MFA Rules not processed for user `78837c5f7b204c9c96f712838f987797`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:20.370991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4c6af236-745b-4026-88b8-af2208230bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:20.422379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:20.466457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 649/1316] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:20.750884 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecf494-b138-4ebe-9fc5-7aa5714a53dd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:20.752450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b9ecf494-b138-4ebe-9fc5-7aa5714a53dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:20.762828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:20.864715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecf494-b138-4ebe-9fc5-7aa5714a53dd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:20.935619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b3adf-98b4-4b99-9146-7b06b7792274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:20.936205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3adf-98b4-4b99-9146-7b06b7792274 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:20.936479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3adf-98b4-4b99-9146-7b06b7792274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:20.936746 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3adf-98b4-4b99-9146-7b06b7792274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:20.938307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c62b3adf-98b4-4b99-9146-7b06b7792274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:20.998517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:21.045097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b3adf-98b4-4b99-9146-7b06b7792274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:21.046561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b3adf-98b4-4b99-9146-7b06b7792274 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:21.064019 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 668/1317] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:23.725648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18c06-2974-4a6f-b6fa-aae70948f6ac None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:23.726208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18c06-2974-4a6f-b6fa-aae70948f6ac None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:23.726208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18c06-2974-4a6f-b6fa-aae70948f6ac None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:23.727594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 650/1318] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:23.732889 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb34d1b-07e9-4077-a48f-993a5e55f56f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:23.733329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb34d1b-07e9-4077-a48f-993a5e55f56f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:23.733454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb34d1b-07e9-4077-a48f-993a5e55f56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:23.800565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5eb34d1b-07e9-4077-a48f-993a5e55f56f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:23.801758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5eb34d1b-07e9-4077-a48f-993a5e55f56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:23.895653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 669/1319] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:24.177390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e18c06-2974-4a6f-b6fa-aae70948f6ac None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:24.178961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-93e18c06-2974-4a6f-b6fa-aae70948f6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:24.280448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e18c06-2974-4a6f-b6fa-aae70948f6ac None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:24.356074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02e97e-21a9-4155-bad0-1bfa3691dd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:24.356677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e97e-21a9-4155-bad0-1bfa3691dd25 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:24.356953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e97e-21a9-4155-bad0-1bfa3691dd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:24.357220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e97e-21a9-4155-bad0-1bfa3691dd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:24.358831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7b02e97e-21a9-4155-bad0-1bfa3691dd25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:24.366836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:24.467164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02e97e-21a9-4155-bad0-1bfa3691dd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:24.468533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02e97e-21a9-4155-bad0-1bfa3691dd25 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:24.486077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 651/1320] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:27.376579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c183-da6a-4122-bbbb-25b5735b6b23 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:27.377085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c183-da6a-4122-bbbb-25b5735b6b23 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:27.377085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c183-da6a-4122-bbbb-25b5735b6b23 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:27.378515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 670/1321] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:27.383764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7475a5-3b49-44cb-aa2f-f2bbd4f33c1f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:27.386977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7475a5-3b49-44cb-aa2f-f2bbd4f33c1f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:27.386977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7475a5-3b49-44cb-aa2f-f2bbd4f33c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:27.450093 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ad7475a5-3b49-44cb-aa2f-f2bbd4f33c1f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:27.451320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ad7475a5-3b49-44cb-aa2f-f2bbd4f33c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:27.510129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:27.549826 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 652/1322] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:27.845411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd2dc9-a2a4-410b-807f-23cba7e8ffa8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:27.845694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd2dc9-a2a4-410b-807f-23cba7e8ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:27.845967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd2dc9-a2a4-410b-807f-23cba7e8ffa8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:27.847403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 671/1323] 10.222.0.58 () {54 vars in 965 bytes} [Tue Aug 23 08:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:27.853588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc772d5-9c1d-4340-8ec6-94fb5feff651 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:27.853977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc772d5-9c1d-4340-8ec6-94fb5feff651 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:27.854258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc772d5-9c1d-4340-8ec6-94fb5feff651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:27.930308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fcc772d5-9c1d-4340-8ec6-94fb5feff651 None None] MFA Rules not processed for user `d99632164b3b4a1f95736a65da217db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:27.931542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fcc772d5-9c1d-4340-8ec6-94fb5feff651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:28.007914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 653/1324] 10.222.0.58 () {58 vars in 1062 bytes} [Tue Aug 23 08:07:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:28.016881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd2dc9-a2a4-410b-807f-23cba7e8ffa8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:28.018961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-84bd2dc9-a2a4-410b-807f-23cba7e8ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:28.038804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:28.123549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd2dc9-a2a4-410b-807f-23cba7e8ffa8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:28.197109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722334e0-9e78-41b8-9ca6-aebef3352e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:28.198086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722334e0-9e78-41b8-9ca6-aebef3352e20 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:28.198367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722334e0-9e78-41b8-9ca6-aebef3352e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:28.198639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722334e0-9e78-41b8-9ca6-aebef3352e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:28.200292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-722334e0-9e78-41b8-9ca6-aebef3352e20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:28.275941 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:28.314161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722334e0-9e78-41b8-9ca6-aebef3352e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:28.315680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722334e0-9e78-41b8-9ca6-aebef3352e20 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:28.333528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 672/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:30.518414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f8ae-f35c-4c42-8756-346c391b33af None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:30.518994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f8ae-f35c-4c42-8756-346c391b33af None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:30.518994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f8ae-f35c-4c42-8756-346c391b33af None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:30.520445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 654/1326] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:30.526127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7c89-70f7-4876-b07a-fd542f7ccbc9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:30.526549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7c89-70f7-4876-b07a-fd542f7ccbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:30.526671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7c89-70f7-4876-b07a-fd542f7ccbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:30.594348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8f2e7c89-70f7-4876-b07a-fd542f7ccbc9 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:30.595579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8f2e7c89-70f7-4876-b07a-fd542f7ccbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:30.700550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 673/1327] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:30.980300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f8ae-f35c-4c42-8756-346c391b33af None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:30.981841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5211f8ae-f35c-4c42-8756-346c391b33af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:31.043322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:31.086551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f8ae-f35c-4c42-8756-346c391b33af None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:31.160096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc6c3e-7ec0-4aef-aa7c-06f142e75c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:31.160696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc6c3e-7ec0-4aef-aa7c-06f142e75c95 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:31.160973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc6c3e-7ec0-4aef-aa7c-06f142e75c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:31.161242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc6c3e-7ec0-4aef-aa7c-06f142e75c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:31.162834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9ddc6c3e-7ec0-4aef-aa7c-06f142e75c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:31.277911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:31.280271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc6c3e-7ec0-4aef-aa7c-06f142e75c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:31.281760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc6c3e-7ec0-4aef-aa7c-06f142e75c95 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:31.300792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 655/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:33.260864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50e1c-ead1-4b43-9dbb-d0f3f3bf7e12 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:33.261379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50e1c-ead1-4b43-9dbb-d0f3f3bf7e12 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:33.261668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50e1c-ead1-4b43-9dbb-d0f3f3bf7e12 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:33.263250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 674/1329] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:33.268769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832865e5-3607-468e-8adb-7e289c53c32a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:33.269194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832865e5-3607-468e-8adb-7e289c53c32a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:33.269306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832865e5-3607-468e-8adb-7e289c53c32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:33.343310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-832865e5-3607-468e-8adb-7e289c53c32a None None] MFA Rules not processed for user `78837c5f7b204c9c96f712838f987797`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:33.344468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-832865e5-3607-468e-8adb-7e289c53c32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:33.442671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 656/1330] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:33 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:07:33.724710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c50e1c-ead1-4b43-9dbb-d0f3f3bf7e12 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:33.726246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-20c50e1c-ead1-4b43-9dbb-d0f3f3bf7e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:33.757160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:33.831279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c50e1c-ead1-4b43-9dbb-d0f3f3bf7e12 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:33.908652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7775937-3ea2-4155-bc31-41c0107acc1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:33.909335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7775937-3ea2-4155-bc31-41c0107acc1b service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:33.909614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7775937-3ea2-4155-bc31-41c0107acc1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:33.909880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7775937-3ea2-4155-bc31-41c0107acc1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:33.911529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f7775937-3ea2-4155-bc31-41c0107acc1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:33.992969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:34.026272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7775937-3ea2-4155-bc31-41c0107acc1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:34.028237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7775937-3ea2-4155-bc31-41c0107acc1b service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:34.048731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 675/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:33 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:36.619722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70883e61-9a8c-413a-819a-ccfb6394361d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:36.620292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70883e61-9a8c-413a-819a-ccfb6394361d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:36.620292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70883e61-9a8c-413a-819a-ccfb6394361d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:36.621651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 657/1332] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:36.626689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb8a58-b1d8-4305-9f6e-104c4f684359 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:36.627108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb8a58-b1d8-4305-9f6e-104c4f684359 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:36.627758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb8a58-b1d8-4305-9f6e-104c4f684359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:36.692456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5acb8a58-b1d8-4305-9f6e-104c4f684359 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:36.693677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5acb8a58-b1d8-4305-9f6e-104c4f684359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:36.787970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 676/1333] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:37.067408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70883e61-9a8c-413a-819a-ccfb6394361d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:37.069009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-70883e61-9a8c-413a-819a-ccfb6394361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:37.137841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:37.172162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70883e61-9a8c-413a-819a-ccfb6394361d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:37.246634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd830c-7528-4220-b811-896bd9e6952c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:37.247308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd830c-7528-4220-b811-896bd9e6952c service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:37.247590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd830c-7528-4220-b811-896bd9e6952c service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:37.247861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd830c-7528-4220-b811-896bd9e6952c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:37.249641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3edd830c-7528-4220-b811-896bd9e6952c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:37.358730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd830c-7528-4220-b811-896bd9e6952c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:37.360116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd830c-7528-4220-b811-896bd9e6952c service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:37.369912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:37.378280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 658/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:39.308500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eaa63-23d6-46d9-b46e-e0d1a3bd0a03 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:39.309029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eaa63-23d6-46d9-b46e-e0d1a3bd0a03 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:39.309029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eaa63-23d6-46d9-b46e-e0d1a3bd0a03 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:39.310402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 677/1335] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:39.316323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02966ed0-f62b-4f5d-a116-0110461f6454 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:39.316749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02966ed0-f62b-4f5d-a116-0110461f6454 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:39.316863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02966ed0-f62b-4f5d-a116-0110461f6454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:39.386889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-02966ed0-f62b-4f5d-a116-0110461f6454 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:39.386889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-02966ed0-f62b-4f5d-a116-0110461f6454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:39.479696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 659/1336] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:39.765279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eaa63-23d6-46d9-b46e-e0d1a3bd0a03 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:39.766985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3f0eaa63-23d6-46d9-b46e-e0d1a3bd0a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:39.808242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:39.870766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eaa63-23d6-46d9-b46e-e0d1a3bd0a03 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:39.940687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac74d05-82bb-483b-b74d-9f524d881b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:39.941277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac74d05-82bb-483b-b74d-9f524d881b35 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:39.941551 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac74d05-82bb-483b-b74d-9f524d881b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:39.941816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac74d05-82bb-483b-b74d-9f524d881b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:39.943646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4ac74d05-82bb-483b-b74d-9f524d881b35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:40.025686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:40.050663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac74d05-82bb-483b-b74d-9f524d881b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:40.052140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac74d05-82bb-483b-b74d-9f524d881b35 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:40.070660 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 678/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:42.561032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b3b70-1489-4b74-b1d6-613047230766 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:42.561603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b3b70-1489-4b74-b1d6-613047230766 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:42.561603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b3b70-1489-4b74-b1d6-613047230766 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:42.563080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 660/1338] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:42.568844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babc9e7-358d-437c-ba04-bb033e59706b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:42.569231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babc9e7-358d-437c-ba04-bb033e59706b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:42.569487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babc9e7-358d-437c-ba04-bb033e59706b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:42.635199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5babc9e7-358d-437c-ba04-bb033e59706b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:42.636417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5babc9e7-358d-437c-ba04-bb033e59706b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:42.730993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 679/1339] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:07:43.019235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b3b70-1489-4b74-b1d6-613047230766 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:43.020822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f50b3b70-1489-4b74-b1d6-613047230766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:43.095686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:43.124233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b3b70-1489-4b74-b1d6-613047230766 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:43.196527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf608-f896-4aef-bb8d-dd3644919308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:43.197127 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf608-f896-4aef-bb8d-dd3644919308 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:43.197406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf608-f896-4aef-bb8d-dd3644919308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:43.197936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf608-f896-4aef-bb8d-dd3644919308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:43.199593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-944bf608-f896-4aef-bb8d-dd3644919308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:43.306559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bf608-f896-4aef-bb8d-dd3644919308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:43.307957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bf608-f896-4aef-bb8d-dd3644919308 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:43.326326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 661/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:47.396760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124c549-c411-44b4-8ed0-74ad4d08cbb4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:47.397298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124c549-c411-44b4-8ed0-74ad4d08cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:47.397298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124c549-c411-44b4-8ed0-74ad4d08cbb4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:47.398657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 680/1341] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:47.404221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de8bb-4dc3-4f7d-9d05-a23850a0cb94 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:47.404639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de8bb-4dc3-4f7d-9d05-a23850a0cb94 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:47.404896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de8bb-4dc3-4f7d-9d05-a23850a0cb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:47.411066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:47.472118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9f7de8bb-4dc3-4f7d-9d05-a23850a0cb94 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:47.473339 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9f7de8bb-4dc3-4f7d-9d05-a23850a0cb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:47.574685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 662/1342] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:47.858244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124c549-c411-44b4-8ed0-74ad4d08cbb4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:47.859815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d124c549-c411-44b4-8ed0-74ad4d08cbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:47.907841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:47.963599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124c549-c411-44b4-8ed0-74ad4d08cbb4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:48.037905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77894e-0a60-441e-8a0f-bd3d41868fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:48.038513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77894e-0a60-441e-8a0f-bd3d41868fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:48.038789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77894e-0a60-441e-8a0f-bd3d41868fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:48.039092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77894e-0a60-441e-8a0f-bd3d41868fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:48.040695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fd77894e-0a60-441e-8a0f-bd3d41868fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:48.134109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:48.150601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77894e-0a60-441e-8a0f-bd3d41868fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:48.152185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77894e-0a60-441e-8a0f-bd3d41868fc8 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:48.170846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 681/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:51.187493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d2ac3-9d46-446a-9aeb-ca6b0119ec51 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:51.188057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d2ac3-9d46-446a-9aeb-ca6b0119ec51 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:51.188057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d2ac3-9d46-446a-9aeb-ca6b0119ec51 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:51.189640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 663/1344] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:51.194567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1dfc2-777d-49b1-8eeb-b4cd174d9cb3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:51.195003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1dfc2-777d-49b1-8eeb-b4cd174d9cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:51.195138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1dfc2-777d-49b1-8eeb-b4cd174d9cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:51.260616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-08b1dfc2-777d-49b1-8eeb-b4cd174d9cb3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:51.261807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-08b1dfc2-777d-49b1-8eeb-b4cd174d9cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:51.360023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 682/1345] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:51.645212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d2ac3-9d46-446a-9aeb-ca6b0119ec51 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:51.646758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-822d2ac3-9d46-446a-9aeb-ca6b0119ec51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:51.740603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:51.758739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d2ac3-9d46-446a-9aeb-ca6b0119ec51 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:51.838809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b469c7-b702-493b-ab4c-c658059233b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:51.839463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b469c7-b702-493b-ab4c-c658059233b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:51.839736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b469c7-b702-493b-ab4c-c658059233b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:51.840003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b469c7-b702-493b-ab4c-c658059233b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:51.841698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-38b469c7-b702-493b-ab4c-c658059233b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:51.950915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b469c7-b702-493b-ab4c-c658059233b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:51.952377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b469c7-b702-493b-ab4c-c658059233b2 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:51.972398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 664/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:07:56.208255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07a248-cfde-4835-87d6-ed0e7418b174 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:56.208787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07a248-cfde-4835-87d6-ed0e7418b174 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:56.208904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07a248-cfde-4835-87d6-ed0e7418b174 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:56.210185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 683/1347] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:07:56.215783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcfd65-d484-411e-ada1-7e51677c193b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:56.216227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcfd65-d484-411e-ada1-7e51677c193b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:56.216323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcfd65-d484-411e-ada1-7e51677c193b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:56.232983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:56.286264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b4bcfd65-d484-411e-ada1-7e51677c193b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:07:56.287515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b4bcfd65-d484-411e-ada1-7e51677c193b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:56.389967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 665/1348] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:07:56.681623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07a248-cfde-4835-87d6-ed0e7418b174 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:07:56.683198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bf07a248-cfde-4835-87d6-ed0e7418b174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:56.739544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:56.785122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07a248-cfde-4835-87d6-ed0e7418b174 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:07:56.856309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb5b1c-31f4-4e7d-a116-7cb5bb20dd12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:07:56.856905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5b1c-31f4-4e7d-a116-7cb5bb20dd12 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:07:56.857177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5b1c-31f4-4e7d-a116-7cb5bb20dd12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:07:56.857438 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5b1c-31f4-4e7d-a116-7cb5bb20dd12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:07:56.859028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dbdb5b1c-31f4-4e7d-a116-7cb5bb20dd12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:07:56.963597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:07:56.967110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb5b1c-31f4-4e7d-a116-7cb5bb20dd12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:07:56.968550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb5b1c-31f4-4e7d-a116-7cb5bb20dd12 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:07:56.987383 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 684/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:01.884453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf6149-fd0c-4483-aade-cf73c12cb057 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:01.884958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf6149-fd0c-4483-aade-cf73c12cb057 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:01.885060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf6149-fd0c-4483-aade-cf73c12cb057 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:01.886474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 666/1350] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:01.894272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04d11-4b1e-4380-820c-9a1e778140c4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:01.895271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04d11-4b1e-4380-820c-9a1e778140c4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:01.895759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04d11-4b1e-4380-820c-9a1e778140c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:01.987699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c3b04d11-4b1e-4380-820c-9a1e778140c4 None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:01.994963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c3b04d11-4b1e-4380-820c-9a1e778140c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:02.120425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 685/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:08:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:06.272247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a459256-b4b0-4644-be06-b0cc0def7e7a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:06.272826 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a459256-b4b0-4644-be06-b0cc0def7e7a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:06.272826 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a459256-b4b0-4644-be06-b0cc0def7e7a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:06.274319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 667/1352] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:06.279604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b741a-2615-4f14-ad48-c0cb851a0d30 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:06.280024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b741a-2615-4f14-ad48-c0cb851a0d30 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:06.280172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b741a-2615-4f14-ad48-c0cb851a0d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:06.340553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:06.364952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a46b741a-2615-4f14-ad48-c0cb851a0d30 None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:06.366286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a46b741a-2615-4f14-ad48-c0cb851a0d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:06.388027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=59 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:06.481835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 686/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:08:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:07.838518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1dfd-04c9-485e-b8f6-1502ca99cd32 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:07.839093 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1dfd-04c9-485e-b8f6-1502ca99cd32 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:07.839093 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1dfd-04c9-485e-b8f6-1502ca99cd32 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:07.840691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 668/1354] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:07.846269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9b099-e725-4de3-b3e7-d8613adc35b8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:07.846724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9b099-e725-4de3-b3e7-d8613adc35b8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:07.846817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9b099-e725-4de3-b3e7-d8613adc35b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:07.921592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-edb9b099-e725-4de3-b3e7-d8613adc35b8 None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:07.922828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-edb9b099-e725-4de3-b3e7-d8613adc35b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:07.967839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:08.033961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 687/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:08:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:11.377313 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2efa-e9e7-46e1-9f78-d51e88d942f5 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:11.377831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2efa-e9e7-46e1-9f78-d51e88d942f5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:11.377831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2efa-e9e7-46e1-9f78-d51e88d942f5 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:11.379225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 669/1356] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:11.384392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a968-660a-43bd-b4ca-2ce8d340117f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:11.384883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a968-660a-43bd-b4ca-2ce8d340117f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:11.384883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a968-660a-43bd-b4ca-2ce8d340117f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:11.461489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4f25a968-660a-43bd-b4ca-2ce8d340117f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:11.462703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4f25a968-660a-43bd-b4ca-2ce8d340117f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:11.570737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 688/1357] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:11.871283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7dd7c-ba00-433e-a2e6-d54a5ceaafae None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:11.871558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7dd7c-ba00-433e-a2e6-d54a5ceaafae None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:11.872007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7dd7c-ba00-433e-a2e6-d54a5ceaafae None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:11.873483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 670/1358] 10.222.0.58 () {54 vars in 962 bytes} [Tue Aug 23 08:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:11.881150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbeb3f-3b61-48b9-a8fe-a4843d940f53 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:11.881444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbeb3f-3b61-48b9-a8fe-a4843d940f53 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:11.881735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbeb3f-3b61-48b9-a8fe-a4843d940f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:11.904113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:11.968816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3ddbeb3f-3b61-48b9-a8fe-a4843d940f53 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:11.970537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3ddbeb3f-3b61-48b9-a8fe-a4843d940f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:12.097985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 689/1359] 10.222.0.58 () {58 vars in 1059 bytes} [Tue Aug 23 08:08:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:12.106522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7dd7c-ba00-433e-a2e6-d54a5ceaafae None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:12.116714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8ff7dd7c-ba00-433e-a2e6-d54a5ceaafae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:12.265336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7dd7c-ba00-433e-a2e6-d54a5ceaafae None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:12.341901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:12.385257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975ea4c-8206-446d-a1b7-cb6e117bfc79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:12.385863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975ea4c-8206-446d-a1b7-cb6e117bfc79 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:12.386142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975ea4c-8206-446d-a1b7-cb6e117bfc79 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:12.386411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975ea4c-8206-446d-a1b7-cb6e117bfc79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:12.388120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4975ea4c-8206-446d-a1b7-cb6e117bfc79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:12.481372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975ea4c-8206-446d-a1b7-cb6e117bfc79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:12.483263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975ea4c-8206-446d-a1b7-cb6e117bfc79 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:12.510247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 671/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:14.619604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c0116-b762-4370-be37-1c49278ca1cd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:14.620123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c0116-b762-4370-be37-1c49278ca1cd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:14.620123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c0116-b762-4370-be37-1c49278ca1cd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:14.621544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 690/1361] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:14.626840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1ddfd-2d35-4d93-b42d-37066407d93d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:14.627288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1ddfd-2d35-4d93-b42d-37066407d93d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:14.627426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1ddfd-2d35-4d93-b42d-37066407d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:14.698844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-25a1ddfd-2d35-4d93-b42d-37066407d93d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:14.700086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-25a1ddfd-2d35-4d93-b42d-37066407d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:14.742734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:14.815165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 672/1362] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:15.101262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c0116-b762-4370-be37-1c49278ca1cd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:15.102884 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ce8c0116-b762-4370-be37-1c49278ca1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:15.182990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:15.276279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c0116-b762-4370-be37-1c49278ca1cd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:15.406167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352090ab-2415-47fc-acb2-59f6bf62b1f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:15.407435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352090ab-2415-47fc-acb2-59f6bf62b1f5 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:15.407777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352090ab-2415-47fc-acb2-59f6bf62b1f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:15.408110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352090ab-2415-47fc-acb2-59f6bf62b1f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:15.409780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-352090ab-2415-47fc-acb2-59f6bf62b1f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:15.433498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:15.512433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352090ab-2415-47fc-acb2-59f6bf62b1f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:15.514179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352090ab-2415-47fc-acb2-59f6bf62b1f5 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:15.544126 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 691/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:17.925156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fc371-4798-44e6-92c6-a62147b107b7 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:17.925696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fc371-4798-44e6-92c6-a62147b107b7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:17.925803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fc371-4798-44e6-92c6-a62147b107b7 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:17.927582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 673/1364] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:17.934993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40d67e-803b-417d-b6de-b55c7350420e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:17.935861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40d67e-803b-417d-b6de-b55c7350420e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:17.935988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40d67e-803b-417d-b6de-b55c7350420e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:18.011149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7d40d67e-803b-417d-b6de-b55c7350420e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:18.012353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7d40d67e-803b-417d-b6de-b55c7350420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:18.114239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:18.123171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 692/1365] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:18.411306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac54e11-d4ef-4611-9edc-074f2c771389 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:18.411657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac54e11-d4ef-4611-9edc-074f2c771389 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:18.411941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac54e11-d4ef-4611-9edc-074f2c771389 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:18.413718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 674/1366] 10.222.0.58 () {54 vars in 962 bytes} [Tue Aug 23 08:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:18.419468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f04e2d-1eb4-468c-a8e0-049eb81c1ea7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:18.419921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f04e2d-1eb4-468c-a8e0-049eb81c1ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:18.425213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f04e2d-1eb4-468c-a8e0-049eb81c1ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:18.501021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-69f04e2d-1eb4-468c-a8e0-049eb81c1ea7 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:18.502249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-69f04e2d-1eb4-468c-a8e0-049eb81c1ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:18.628215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 693/1367] 10.222.0.58 () {58 vars in 1059 bytes} [Tue Aug 23 08:08:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:18.636021 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac54e11-d4ef-4611-9edc-074f2c771389 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:18.637618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aac54e11-d4ef-4611-9edc-074f2c771389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:18.801878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac54e11-d4ef-4611-9edc-074f2c771389 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:18.824898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:18.931198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de37338-f796-4422-8541-0c875092f23d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:18.931887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de37338-f796-4422-8541-0c875092f23d service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:18.932152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de37338-f796-4422-8541-0c875092f23d service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:18.932329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de37338-f796-4422-8541-0c875092f23d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:18.933908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4de37338-f796-4422-8541-0c875092f23d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:19.025407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de37338-f796-4422-8541-0c875092f23d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:19.027181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de37338-f796-4422-8541-0c875092f23d service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:19.045688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 675/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:08:19.701706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26c12-d6ee-4e18-ba7f-72090a6f760c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:19.702279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26c12-d6ee-4e18-ba7f-72090a6f760c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:19.702279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26c12-d6ee-4e18-ba7f-72090a6f760c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:19.703707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 694/1369] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:19.709083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff409f-aba8-4b65-998a-9e167c43f6ad None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:19.709372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff409f-aba8-4b65-998a-9e167c43f6ad None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:19.709645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff409f-aba8-4b65-998a-9e167c43f6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:19.742508 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:19.780838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2bff409f-aba8-4b65-998a-9e167c43f6ad None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:19.782376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2bff409f-aba8-4b65-998a-9e167c43f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:19.889902 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 676/1370] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:20.471184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7d668-fa0e-49a9-9aa6-ba5773ab766b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:20.471569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7d668-fa0e-49a9-9aa6-ba5773ab766b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:20.471938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7d668-fa0e-49a9-9aa6-ba5773ab766b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:20.473512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 695/1371] 10.222.0.58 () {54 vars in 964 bytes} [Tue Aug 23 08:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:20.478512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932b5c-c0bd-4410-9f40-b43acd9aaa20 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:20.479143 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932b5c-c0bd-4410-9f40-b43acd9aaa20 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:20.479143 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5932b5c-c0bd-4410-9f40-b43acd9aaa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:20.560529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c5932b5c-c0bd-4410-9f40-b43acd9aaa20 None None] MFA Rules not processed for user `6adffd03ce3a4ebfb2d7c5df3a2ca293`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:20.561797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c5932b5c-c0bd-4410-9f40-b43acd9aaa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:20.613216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:20.643575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 677/1372] 10.222.0.58 () {58 vars in 1061 bytes} [Tue Aug 23 08:08:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:20.650999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7d668-fa0e-49a9-9aa6-ba5773ab766b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:20.652547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0ec7d668-fa0e-49a9-9aa6-ba5773ab766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:20.705588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:20.765963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7d668-fa0e-49a9-9aa6-ba5773ab766b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:20.845685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373abebf-7f5d-4971-9997-07138a0b3091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:20.846343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373abebf-7f5d-4971-9997-07138a0b3091 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:20.846614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373abebf-7f5d-4971-9997-07138a0b3091 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:20.846876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373abebf-7f5d-4971-9997-07138a0b3091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:20.848486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-373abebf-7f5d-4971-9997-07138a0b3091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:20.946003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:20.962041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373abebf-7f5d-4971-9997-07138a0b3091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:20.963714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373abebf-7f5d-4971-9997-07138a0b3091 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:20.991523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 696/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:21.216989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10610b-cbbf-4871-aa04-cc11f3a341cf None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:21.217214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10610b-cbbf-4871-aa04-cc11f3a341cf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:21.217493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10610b-cbbf-4871-aa04-cc11f3a341cf None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:21.226981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 678/1374] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:21.227230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fb45-7d67-409c-a5e6-5c7a4f8c7984 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:21.227230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fb45-7d67-409c-a5e6-5c7a4f8c7984 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:21.227230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5fb45-7d67-409c-a5e6-5c7a4f8c7984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:21.294503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-59e5fb45-7d67-409c-a5e6-5c7a4f8c7984 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:21.295930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-59e5fb45-7d67-409c-a5e6-5c7a4f8c7984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:21.391882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 697/1375] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:21.682667 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10610b-cbbf-4871-aa04-cc11f3a341cf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:21.684235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1c10610b-cbbf-4871-aa04-cc11f3a341cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:21.834304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10610b-cbbf-4871-aa04-cc11f3a341cf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:21.885818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:21.948444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2572113-0b2e-49fd-adda-28d3dc22e609 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:21.949002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572113-0b2e-49fd-adda-28d3dc22e609 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:21.949279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572113-0b2e-49fd-adda-28d3dc22e609 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:21.949548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2572113-0b2e-49fd-adda-28d3dc22e609 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:21.951214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f2572113-0b2e-49fd-adda-28d3dc22e609 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:22.040669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2572113-0b2e-49fd-adda-28d3dc22e609 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:22.042297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2572113-0b2e-49fd-adda-28d3dc22e609 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:22.061620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 679/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:24.134296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d422700-0359-4536-acb1-53bea747137b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:24.134766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d422700-0359-4536-acb1-53bea747137b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:24.134766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d422700-0359-4536-acb1-53bea747137b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:24.136328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 698/1377] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:24.142122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81275e94-fba3-4fd0-8188-99ef27074647 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:24.142547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81275e94-fba3-4fd0-8188-99ef27074647 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:24.142655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81275e94-fba3-4fd0-8188-99ef27074647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:24.213682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-81275e94-fba3-4fd0-8188-99ef27074647 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:24.214896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-81275e94-fba3-4fd0-8188-99ef27074647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:24.224109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:24.331810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 680/1378] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:24.627237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d422700-0359-4536-acb1-53bea747137b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:24.628849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7d422700-0359-4536-acb1-53bea747137b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:24.720824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:24.830245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d422700-0359-4536-acb1-53bea747137b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:24.950469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a45bde-5b46-4d41-b58e-7ea1be7bc2c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:24.951112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45bde-5b46-4d41-b58e-7ea1be7bc2c8 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:24.951406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45bde-5b46-4d41-b58e-7ea1be7bc2c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:24.951666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45bde-5b46-4d41-b58e-7ea1be7bc2c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:24.953297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-61a45bde-5b46-4d41-b58e-7ea1be7bc2c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:24.973252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:25.050771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a45bde-5b46-4d41-b58e-7ea1be7bc2c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:25.052621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a45bde-5b46-4d41-b58e-7ea1be7bc2c8 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:25.072484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 699/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:27.150553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a7782-4604-4cc9-9628-abb5c6ec8973 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:27.151049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a7782-4604-4cc9-9628-abb5c6ec8973 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:27.151148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a7782-4604-4cc9-9628-abb5c6ec8973 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:27.152455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 681/1380] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:27.157927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1d28a-40f4-4d67-879c-554a663a72b8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:27.158330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1d28a-40f4-4d67-879c-554a663a72b8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:27.158485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1d28a-40f4-4d67-879c-554a663a72b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:27.226219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1eb1d28a-40f4-4d67-879c-554a663a72b8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:27.227442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1eb1d28a-40f4-4d67-879c-554a663a72b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:27.322310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:27.329147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 700/1381] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:08:27.620876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a7782-4604-4cc9-9628-abb5c6ec8973 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:27.622395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f36a7782-4604-4cc9-9628-abb5c6ec8973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:27.776468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:27.794973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a7782-4604-4cc9-9628-abb5c6ec8973 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:27.915073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0971619-cda5-4bb5-a519-fe3c3d92b241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:27.915688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971619-cda5-4bb5-a519-fe3c3d92b241 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:27.915968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971619-cda5-4bb5-a519-fe3c3d92b241 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:27.916241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971619-cda5-4bb5-a519-fe3c3d92b241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:27.917847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d0971619-cda5-4bb5-a519-fe3c3d92b241 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:28.014887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0971619-cda5-4bb5-a519-fe3c3d92b241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:28.016754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0971619-cda5-4bb5-a519-fe3c3d92b241 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:28.027637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:28.039118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 682/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:30.120177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95b4b8-b935-443d-a799-e425182fda3b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:30.120709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95b4b8-b935-443d-a799-e425182fda3b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:30.120891 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95b4b8-b935-443d-a799-e425182fda3b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:30.122653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 701/1383] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:30.129836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6156e-efeb-4411-bcbd-b07b625ff6df None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:30.130246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6156e-efeb-4411-bcbd-b07b625ff6df None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:30.130410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6156e-efeb-4411-bcbd-b07b625ff6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:30.207732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d6c6156e-efeb-4411-bcbd-b07b625ff6df None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:30.209017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d6c6156e-efeb-4411-bcbd-b07b625ff6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:30.309258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 683/1384] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:30.596575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95b4b8-b935-443d-a799-e425182fda3b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:30.598222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6d95b4b8-b935-443d-a799-e425182fda3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:30.744923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95b4b8-b935-443d-a799-e425182fda3b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:30.821068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:30.878165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1c2c1-aa1c-43b2-a7a2-556c0d88c1c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:30.878714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1c2c1-aa1c-43b2-a7a2-556c0d88c1c8 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:30.879034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1c2c1-aa1c-43b2-a7a2-556c0d88c1c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:30.879414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1c2c1-aa1c-43b2-a7a2-556c0d88c1c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:30.881062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-baf1c2c1-aa1c-43b2-a7a2-556c0d88c1c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:30.974516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1c2c1-aa1c-43b2-a7a2-556c0d88c1c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:30.976227 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1c2c1-aa1c-43b2-a7a2-556c0d88c1c8 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:30.995627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 702/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:08:32.896371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a188c-ffe7-4764-8175-5bb6e8542932 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:32.896896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a188c-ffe7-4764-8175-5bb6e8542932 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:32.896896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a188c-ffe7-4764-8175-5bb6e8542932 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:32.898223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 684/1386] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:32.903533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc0d5f-14a0-4b67-b48d-f1b8ecbcd14f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:32.904058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc0d5f-14a0-4b67-b48d-f1b8ecbcd14f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:32.904058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc0d5f-14a0-4b67-b48d-f1b8ecbcd14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:32.970095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-49bc0d5f-14a0-4b67-b48d-f1b8ecbcd14f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:32.971381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-49bc0d5f-14a0-4b67-b48d-f1b8ecbcd14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:33.003816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:33.071799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 703/1387] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:33.359700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a188c-ffe7-4764-8175-5bb6e8542932 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:33.361326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-793a188c-ffe7-4764-8175-5bb6e8542932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:33.444883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:33.529701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a188c-ffe7-4764-8175-5bb6e8542932 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:33.657165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9caa73-915e-49bc-961b-0817e01273da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:33.657716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9caa73-915e-49bc-961b-0817e01273da service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:33.658108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9caa73-915e-49bc-961b-0817e01273da service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:33.658333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9caa73-915e-49bc-961b-0817e01273da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:33.659975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ad9caa73-915e-49bc-961b-0817e01273da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:33.700106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:33.758608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9caa73-915e-49bc-961b-0817e01273da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:33.760389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9caa73-915e-49bc-961b-0817e01273da service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:33.779778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 685/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:35.930670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2984-a200-445a-b2ad-25cd1d6a9844 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:35.931233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2984-a200-445a-b2ad-25cd1d6a9844 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:35.931349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2984-a200-445a-b2ad-25cd1d6a9844 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:35.932605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 704/1389] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:35.937964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033180bc-228b-43ce-b227-c5a22c46ef8b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:35.938404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033180bc-228b-43ce-b227-c5a22c46ef8b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:35.938510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033180bc-228b-43ce-b227-c5a22c46ef8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:36.005655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-033180bc-228b-43ce-b227-c5a22c46ef8b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:36.007424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-033180bc-228b-43ce-b227-c5a22c46ef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:36.107069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 686/1390] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:36.405704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf2984-a200-445a-b2ad-25cd1d6a9844 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:36.407289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c5cf2984-a200-445a-b2ad-25cd1d6a9844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:36.428235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899115b1-feb2-41f0-a52d-ccd806b0f7bd None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:36.428555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899115b1-feb2-41f0-a52d-ccd806b0f7bd None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:36.428999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899115b1-feb2-41f0-a52d-ccd806b0f7bd None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:36.430185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 687/1391] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 08:08:36.435310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ed60-1e20-44ca-968f-fdffe0dc4034 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:36.435592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ed60-1e20-44ca-968f-fdffe0dc4034 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:36.435869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ed60-1e20-44ca-968f-fdffe0dc4034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:36.452386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:36.507054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8da1ed60-1e20-44ca-968f-fdffe0dc4034 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:36.508264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8da1ed60-1e20-44ca-968f-fdffe0dc4034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:36.558007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf2984-a200-445a-b2ad-25cd1d6a9844 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:36.575267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:36.609519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 688/1392] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:08:36.674483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326d228-0620-4629-8b48-80a8f374ab15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:36.675119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326d228-0620-4629-8b48-80a8f374ab15 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:36.675421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326d228-0620-4629-8b48-80a8f374ab15 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:36.675672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326d228-0620-4629-8b48-80a8f374ab15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:36.677288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5326d228-0620-4629-8b48-80a8f374ab15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:36.767811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326d228-0620-4629-8b48-80a8f374ab15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:36.769428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326d228-0620-4629-8b48-80a8f374ab15 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:36.791007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 705/1393] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:36.913007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccb67c-235b-41ea-9790-7a878b7a5948 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:36.913287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccb67c-235b-41ea-9790-7a878b7a5948 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:36.913607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccb67c-235b-41ea-9790-7a878b7a5948 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:36.915002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 689/1394] 10.222.0.58 () {54 vars in 967 bytes} [Tue Aug 23 08:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:36.920033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892954bb-9074-44d2-a263-87b6b331a3dd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:36.920376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892954bb-9074-44d2-a263-87b6b331a3dd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:36.920901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892954bb-9074-44d2-a263-87b6b331a3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:36.945932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:37.007639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-892954bb-9074-44d2-a263-87b6b331a3dd None None] MFA Rules not processed for user `5e9526306889402c83f2d33b340c2dac`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:37.009506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-892954bb-9074-44d2-a263-87b6b331a3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:37.095928 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 706/1395] 10.222.0.58 () {58 vars in 1064 bytes} [Tue Aug 23 08:08:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:37.103453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccb67c-235b-41ea-9790-7a878b7a5948 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:37.105161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-46ccb67c-235b-41ea-9790-7a878b7a5948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:37.203796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:37.210668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccb67c-235b-41ea-9790-7a878b7a5948 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:37.286564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f5c72-983e-473a-a16a-b0683a63a46d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e9526306889402c83f2d33b340c2dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:37.287201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5c72-983e-473a-a16a-b0683a63a46d service glance] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:37.287483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5c72-983e-473a-a16a-b0683a63a46d service glance] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:37.287756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5c72-983e-473a-a16a-b0683a63a46d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:37.289349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-769f5c72-983e-473a-a16a-b0683a63a46d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:37.401072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f5c72-983e-473a-a16a-b0683a63a46d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:37.402512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f5c72-983e-473a-a16a-b0683a63a46d service glance] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:37.422185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 690/1396] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:38.372087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd55ef-f361-4f01-9fe3-a923de6e1cd6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:38.372564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd55ef-f361-4f01-9fe3-a923de6e1cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:38.372665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd55ef-f361-4f01-9fe3-a923de6e1cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:38.439515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e7fd55ef-f361-4f01-9fe3-a923de6e1cd6 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:38.440727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e7fd55ef-f361-4f01-9fe3-a923de6e1cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:38.501282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:38.529771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 707/1397] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:08:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:38.576743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4a144-430e-4421-9bd5-0b4150c88362 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:38.577234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4a144-430e-4421-9bd5-0b4150c88362 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:38.578023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4a144-430e-4421-9bd5-0b4150c88362 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:38.579441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 691/1398] 10.222.0.58 () {54 vars in 943 bytes} [Tue Aug 23 08:08:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:38.585922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662511e-9b1e-45d6-89aa-b384a8bc527a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:38.586216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662511e-9b1e-45d6-89aa-b384a8bc527a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:38.586496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662511e-9b1e-45d6-89aa-b384a8bc527a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:38.663248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0662511e-9b1e-45d6-89aa-b384a8bc527a None None] MFA Rules not processed for user `d4705311aa574da7b9f1b9d31c40cd72`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:38.664461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0662511e-9b1e-45d6-89aa-b384a8bc527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:38.771834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 708/1399] 10.222.0.58 () {58 vars in 1040 bytes} [Tue Aug 23 08:08:38 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:38.782054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4a144-430e-4421-9bd5-0b4150c88362 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:38.783781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0fa4a144-430e-4421-9bd5-0b4150c88362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:38.807377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:38.844463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9094f-be89-4d36-aebb-9691d11feedb None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:38.844751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9094f-be89-4d36-aebb-9691d11feedb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:38.845026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9094f-be89-4d36-aebb-9691d11feedb None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:38.846384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 709/1400] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 08:08:38.851709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec107d40-666e-40ef-84e2-c8f0a5ae2561 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:38.851992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec107d40-666e-40ef-84e2-c8f0a5ae2561 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:38.852268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec107d40-666e-40ef-84e2-c8f0a5ae2561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:38.897597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:38.918501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ec107d40-666e-40ef-84e2-c8f0a5ae2561 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:38.919675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ec107d40-666e-40ef-84e2-c8f0a5ae2561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:38.928264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4a144-430e-4421-9bd5-0b4150c88362 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:39.018419 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 710/1401] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:08:39.034903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddf45c-8a7a-4bc0-9854-89ef70e51212 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:39.035537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddf45c-8a7a-4bc0-9854-89ef70e51212 service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:39.035809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddf45c-8a7a-4bc0-9854-89ef70e51212 service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:39.036072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddf45c-8a7a-4bc0-9854-89ef70e51212 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:39.037703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6eddf45c-8a7a-4bc0-9854-89ef70e51212 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:39.044963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:39.121363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddf45c-8a7a-4bc0-9854-89ef70e51212 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:39.122970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddf45c-8a7a-4bc0-9854-89ef70e51212 service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:39.130909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 692/1402] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:08:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:08:39.306494 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec107d40-666e-40ef-84e2-c8f0a5ae2561 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:39.307744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ec107d40-666e-40ef-84e2-c8f0a5ae2561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:39.423637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec107d40-666e-40ef-84e2-c8f0a5ae2561 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:39.427269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:39.562425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3e8df-9131-4bb0-a386-efcb4fcc222f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:39.563305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3e8df-9131-4bb0-a386-efcb4fcc222f service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:39.563589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3e8df-9131-4bb0-a386-efcb4fcc222f service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:39.563857 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3e8df-9131-4bb0-a386-efcb4fcc222f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:39.565473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5ae3e8df-9131-4bb0-a386-efcb4fcc222f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:39.665743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3e8df-9131-4bb0-a386-efcb4fcc222f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:39.667709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3e8df-9131-4bb0-a386-efcb4fcc222f service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:39.686160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 711/1403] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:41.759775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af85c1a-e01e-4fe2-8948-b8c283a01dd3 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:41.759775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af85c1a-e01e-4fe2-8948-b8c283a01dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:41.759775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af85c1a-e01e-4fe2-8948-b8c283a01dd3 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:41.760692 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 693/1404] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:41.767596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d594-51f1-48fd-8c8a-eb5189197a9b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:41.768037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d594-51f1-48fd-8c8a-eb5189197a9b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:41.768283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d594-51f1-48fd-8c8a-eb5189197a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:41.781332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:41.850343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a5d5d594-51f1-48fd-8c8a-eb5189197a9b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:41.887412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a5d5d594-51f1-48fd-8c8a-eb5189197a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:41.991715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 712/1405] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:42.327922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af85c1a-e01e-4fe2-8948-b8c283a01dd3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:42.329439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9af85c1a-e01e-4fe2-8948-b8c283a01dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:42.528671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af85c1a-e01e-4fe2-8948-b8c283a01dd3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:42.532657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:42.664053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24722333-2d2c-4afc-9011-3b8266c6ec50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:42.667146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24722333-2d2c-4afc-9011-3b8266c6ec50 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:42.667146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24722333-2d2c-4afc-9011-3b8266c6ec50 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:42.667146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24722333-2d2c-4afc-9011-3b8266c6ec50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:42.667545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-24722333-2d2c-4afc-9011-3b8266c6ec50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:42.769440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24722333-2d2c-4afc-9011-3b8266c6ec50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:42.771146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24722333-2d2c-4afc-9011-3b8266c6ec50 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:42.791419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 694/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:44.888267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946be283-772a-4278-a969-924352bb19be None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:44.888835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946be283-772a-4278-a969-924352bb19be None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:44.888835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946be283-772a-4278-a969-924352bb19be None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:44.890804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 713/1407] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:08:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:08:44.896262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e219a-594c-47ad-8560-72f7d89ec742 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:44.896661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e219a-594c-47ad-8560-72f7d89ec742 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:44.896819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e219a-594c-47ad-8560-72f7d89ec742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:44.906630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:44.969790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ca2e219a-594c-47ad-8560-72f7d89ec742 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:08:44.971049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ca2e219a-594c-47ad-8560-72f7d89ec742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:45.075870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 695/1408] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:08:45.373850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946be283-772a-4278-a969-924352bb19be None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:08:45.375943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-946be283-772a-4278-a969-924352bb19be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:45.481430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:45.550458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946be283-772a-4278-a969-924352bb19be None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:08:45.677708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff167e81-6752-409f-99a3-8f3254abcb36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:08:45.678322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff167e81-6752-409f-99a3-8f3254abcb36 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:08:45.678600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff167e81-6752-409f-99a3-8f3254abcb36 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:08:45.678877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff167e81-6752-409f-99a3-8f3254abcb36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:08:45.680523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ff167e81-6752-409f-99a3-8f3254abcb36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:08:45.739425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:45.784590 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff167e81-6752-409f-99a3-8f3254abcb36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:08:45.786248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff167e81-6752-409f-99a3-8f3254abcb36 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:08:45.807726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 714/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:08:55.073847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=62 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:08:55.807576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=23 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:02.692899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81439d67-24ef-4c00-90f0-e327a757d4f6 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:02.693473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81439d67-24ef-4c00-90f0-e327a757d4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:02.693473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81439d67-24ef-4c00-90f0-e327a757d4f6 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:02.694796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 696/1410] 10.222.0.58 () {54 vars in 918 bytes} [Tue Aug 23 08:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:02.700249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121e8c5-c065-4a16-8722-e1d6e0cae64e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:02.700685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121e8c5-c065-4a16-8722-e1d6e0cae64e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:02.700781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121e8c5-c065-4a16-8722-e1d6e0cae64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:02.776174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c121e8c5-c065-4a16-8722-e1d6e0cae64e None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:02.777423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c121e8c5-c065-4a16-8722-e1d6e0cae64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:02.886409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 715/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Tue Aug 23 08:09:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:02.901109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720e16-3dcd-4a69-8abe-7b96b3de452a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:02.901396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720e16-3dcd-4a69-8abe-7b96b3de452a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:02.901670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720e16-3dcd-4a69-8abe-7b96b3de452a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:02.903038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 697/1412] 10.222.0.58 () {54 vars in 918 bytes} [Tue Aug 23 08:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:02.907739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c063d00-af35-4280-a4c8-1ff443b320d6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:02.908029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c063d00-af35-4280-a4c8-1ff443b320d6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:02.908301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c063d00-af35-4280-a4c8-1ff443b320d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:02.979011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:02.981496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7c063d00-af35-4280-a4c8-1ff443b320d6 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:02.982687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c063d00-af35-4280-a4c8-1ff443b320d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:03.097603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 716/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Tue Aug 23 08:09:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:09:05.458860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eecdc-6c75-47bb-825a-1da4d121cad9 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:05.459434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eecdc-6c75-47bb-825a-1da4d121cad9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:05.459434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eecdc-6c75-47bb-825a-1da4d121cad9 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:05.460824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:05.466209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd064-b639-4fa2-883c-f59223282dfa None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:05.466658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd064-b639-4fa2-883c-f59223282dfa None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:05.467379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd064-b639-4fa2-883c-f59223282dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:05.532616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ce6dd064-b639-4fa2-883c-f59223282dfa None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:05.533835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ce6dd064-b639-4fa2-883c-f59223282dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:05.587923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:05.628804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:05.919830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39eecdc-6c75-47bb-825a-1da4d121cad9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:05.921680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a39eecdc-6c75-47bb-825a-1da4d121cad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:06.066282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39eecdc-6c75-47bb-825a-1da4d121cad9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:06.139920 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:06.179349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d9711-c486-4942-b987-4dde004474e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:06.179942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d9711-c486-4942-b987-4dde004474e6 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:06.180216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d9711-c486-4942-b987-4dde004474e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:06.180481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d9711-c486-4942-b987-4dde004474e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:06.182073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9f5d9711-c486-4942-b987-4dde004474e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:06.269182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d9711-c486-4942-b987-4dde004474e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:06.270763 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d9711-c486-4942-b987-4dde004474e6 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:06.288211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 699/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:07.176511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a364-2b4f-4745-91fc-07e6ee065bb4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:07.176980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a364-2b4f-4745-91fc-07e6ee065bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:07.177097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a364-2b4f-4745-91fc-07e6ee065bb4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:07.178483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:07.183346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8094f-1839-48f7-83a7-a4c1560c90c3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:07.183761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8094f-1839-48f7-83a7-a4c1560c90c3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:07.183885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8094f-1839-48f7-83a7-a4c1560c90c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:07.261393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3dd8094f-1839-48f7-83a7-a4c1560c90c3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:07.262599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3dd8094f-1839-48f7-83a7-a4c1560c90c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:07.289520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:07.360005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:07.508573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1a364-2b4f-4745-91fc-07e6ee065bb4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:07.510102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-69a1a364-2b4f-4745-91fc-07e6ee065bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:07.653613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1a364-2b4f-4745-91fc-07e6ee065bb4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:07.693116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:07.767717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cb222-4dec-44f0-8a28-58639fb2551e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:07.768246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb222-4dec-44f0-8a28-58639fb2551e service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:07.768516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb222-4dec-44f0-8a28-58639fb2551e service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:07.768778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cb222-4dec-44f0-8a28-58639fb2551e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:07.770492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-087cb222-4dec-44f0-8a28-58639fb2551e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:07.860360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb222-4dec-44f0-8a28-58639fb2551e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:07.861967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cb222-4dec-44f0-8a28-58639fb2551e service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:07.879931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 719/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:08.759488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8faa6-da86-4419-84fe-d226610947fd None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:08.760129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8faa6-da86-4419-84fe-d226610947fd None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:08.760129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8faa6-da86-4419-84fe-d226610947fd None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:08.761390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:08.766549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd76c-4a54-4b72-a38d-498d29181516 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:08.767030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd76c-4a54-4b72-a38d-498d29181516 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:08.767141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd76c-4a54-4b72-a38d-498d29181516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:08.821252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:08.842787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-510dd76c-4a54-4b72-a38d-498d29181516 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:08.844142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-510dd76c-4a54-4b72-a38d-498d29181516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:08.940413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:09.090868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8faa6-da86-4419-84fe-d226610947fd None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:09.092422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8de8faa6-da86-4419-84fe-d226610947fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:09.238136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8faa6-da86-4419-84fe-d226610947fd None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:09.251194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:09.350428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245545a-8004-4e84-ae6a-128fef4b2cb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:09.350979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245545a-8004-4e84-ae6a-128fef4b2cb3 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:09.351261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245545a-8004-4e84-ae6a-128fef4b2cb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:09.351532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245545a-8004-4e84-ae6a-128fef4b2cb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:09.353124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d245545a-8004-4e84-ae6a-128fef4b2cb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:09.440792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245545a-8004-4e84-ae6a-128fef4b2cb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:09.442387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245545a-8004-4e84-ae6a-128fef4b2cb3 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:09.460195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 702/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:10.317242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d57a4-4a56-4281-8c92-03dc6609368d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:10.317796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d57a4-4a56-4281-8c92-03dc6609368d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:10.317913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d57a4-4a56-4281-8c92-03dc6609368d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:10.319220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:10.324617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab927c-4bbd-41b0-b9bb-db5e61845009 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:10.325066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab927c-4bbd-41b0-b9bb-db5e61845009 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:10.325196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab927c-4bbd-41b0-b9bb-db5e61845009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:10.345581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:10.401615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-55ab927c-4bbd-41b0-b9bb-db5e61845009 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:10.402805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-55ab927c-4bbd-41b0-b9bb-db5e61845009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:10.501004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:10.650239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d57a4-4a56-4281-8c92-03dc6609368d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:10.651841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c9d57a4-4a56-4281-8c92-03dc6609368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:10.765998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:10.802444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d57a4-4a56-4281-8c92-03dc6609368d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:10.914812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517735f-f61e-4830-b2e7-c7655f033810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:10.915451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517735f-f61e-4830-b2e7-c7655f033810 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:10.915729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517735f-f61e-4830-b2e7-c7655f033810 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:10.915999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517735f-f61e-4830-b2e7-c7655f033810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:10.917598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0517735f-f61e-4830-b2e7-c7655f033810 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:10.993873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:11.007535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517735f-f61e-4830-b2e7-c7655f033810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:11.009134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517735f-f61e-4830-b2e7-c7655f033810 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:11.027192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 722/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:11.882809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be89c8-5cd8-4d85-ac2b-5b83d68f59cc None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:11.883399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be89c8-5cd8-4d85-ac2b-5b83d68f59cc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:11.883399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be89c8-5cd8-4d85-ac2b-5b83d68f59cc None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:11.884733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:11.890314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bc0fa-506d-4746-acee-9a5d922a40e0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:11.890752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bc0fa-506d-4746-acee-9a5d922a40e0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:11.890859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bc0fa-506d-4746-acee-9a5d922a40e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:11.966380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b59bc0fa-506d-4746-acee-9a5d922a40e0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:11.967628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b59bc0fa-506d-4746-acee-9a5d922a40e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:12.065433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:12.217340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be89c8-5cd8-4d85-ac2b-5b83d68f59cc None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:12.218874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-09be89c8-5cd8-4d85-ac2b-5b83d68f59cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:12.301243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:12.362177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be89c8-5cd8-4d85-ac2b-5b83d68f59cc None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:12.471459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccbd60-a2df-4368-9a2b-0dbe0565c80c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:12.472055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccbd60-a2df-4368-9a2b-0dbe0565c80c service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:12.472327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccbd60-a2df-4368-9a2b-0dbe0565c80c service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:12.472593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccbd60-a2df-4368-9a2b-0dbe0565c80c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:12.474163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-21ccbd60-a2df-4368-9a2b-0dbe0565c80c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:12.521063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:12.563186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccbd60-a2df-4368-9a2b-0dbe0565c80c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:12.564846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccbd60-a2df-4368-9a2b-0dbe0565c80c service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:12.583510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 705/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:13.470207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f22796-522b-4a8d-844d-95f5cbf19fb8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:13.470731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f22796-522b-4a8d-844d-95f5cbf19fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:13.470731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f22796-522b-4a8d-844d-95f5cbf19fb8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:13.472118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 08:09:13.477131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ce80-af4f-4f16-92a3-5d7d0699a427 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:13.477566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ce80-af4f-4f16-92a3-5d7d0699a427 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:13.477690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ce80-af4f-4f16-92a3-5d7d0699a427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:13.554985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8581ce80-af4f-4f16-92a3-5d7d0699a427 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:13.556483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8581ce80-af4f-4f16-92a3-5d7d0699a427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:13.652169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:13.802792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f22796-522b-4a8d-844d-95f5cbf19fb8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:13.804383 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c6f22796-522b-4a8d-844d-95f5cbf19fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:13.853302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:13.949918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f22796-522b-4a8d-844d-95f5cbf19fb8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:14.061964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b47c7-27cf-435e-be59-daeb07b5aced service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:14.062488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b47c7-27cf-435e-be59-daeb07b5aced service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:14.062766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b47c7-27cf-435e-be59-daeb07b5aced service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:14.063069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b47c7-27cf-435e-be59-daeb07b5aced service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:14.064676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a66b47c7-27cf-435e-be59-daeb07b5aced service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:14.072971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:14.155252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b47c7-27cf-435e-be59-daeb07b5aced service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:14.156844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b47c7-27cf-435e-be59-daeb07b5aced service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:14.174641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 725/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:15.052293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac48b4-fd72-44e6-adc9-c9365633c948 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:15.052809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac48b4-fd72-44e6-adc9-c9365633c948 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:15.052809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac48b4-fd72-44e6-adc9-c9365633c948 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:15.054090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:15.059110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e5b7e-33bc-44e2-a2aa-378fd0a0b7bb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:15.059532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e5b7e-33bc-44e2-a2aa-378fd0a0b7bb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:15.059628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e5b7e-33bc-44e2-a2aa-378fd0a0b7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:15.134609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-933e5b7e-33bc-44e2-a2aa-378fd0a0b7bb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:15.135856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-933e5b7e-33bc-44e2-a2aa-378fd0a0b7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:15.203675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:15.231988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:15.382251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac48b4-fd72-44e6-adc9-c9365633c948 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:15.383741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3bac48b4-fd72-44e6-adc9-c9365633c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:15.401934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:15.530875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac48b4-fd72-44e6-adc9-c9365633c948 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:15.619432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:15.646298 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e8dff-62d1-427b-94ab-46d6fbb5d04c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:15.646832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8dff-62d1-427b-94ab-46d6fbb5d04c service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:15.647146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8dff-62d1-427b-94ab-46d6fbb5d04c service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:15.647418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8dff-62d1-427b-94ab-46d6fbb5d04c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:15.649024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a44e8dff-62d1-427b-94ab-46d6fbb5d04c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:15.740768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e8dff-62d1-427b-94ab-46d6fbb5d04c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:15.742434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e8dff-62d1-427b-94ab-46d6fbb5d04c service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:15.761097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 708/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:16.616230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc25a6-105e-4df8-984a-f52e1fb3b77a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:16.616797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc25a6-105e-4df8-984a-f52e1fb3b77a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:16.616797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc25a6-105e-4df8-984a-f52e1fb3b77a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:16.618098 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:16.623143 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bed0f-0d26-455b-91bd-43702a6fab00 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:16.625322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bed0f-0d26-455b-91bd-43702a6fab00 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:16.625322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bed0f-0d26-455b-91bd-43702a6fab00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:16.701962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-672bed0f-0d26-455b-91bd-43702a6fab00 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:16.703194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-672bed0f-0d26-455b-91bd-43702a6fab00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:16.744430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:16.800060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:16.947850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc25a6-105e-4df8-984a-f52e1fb3b77a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:16.949348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a6cc25a6-105e-4df8-984a-f52e1fb3b77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:17.094655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc25a6-105e-4df8-984a-f52e1fb3b77a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:17.147720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:17.206991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d3a6c-64e3-454a-9b39-64b136294e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:17.207627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3a6c-64e3-454a-9b39-64b136294e05 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:17.207903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3a6c-64e3-454a-9b39-64b136294e05 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:17.208167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3a6c-64e3-454a-9b39-64b136294e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:17.209734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ef0d3a6c-64e3-454a-9b39-64b136294e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:17.297794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d3a6c-64e3-454a-9b39-64b136294e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:17.299459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d3a6c-64e3-454a-9b39-64b136294e05 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:17.317626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 728/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:18.160612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cefab5-fc7f-43df-be33-f80c9dc45d3a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:18.161129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cefab5-fc7f-43df-be33-f80c9dc45d3a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:18.161129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cefab5-fc7f-43df-be33-f80c9dc45d3a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:18.162480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:18.167332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464301bb-2132-4161-8b18-0bf70f9b6631 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:18.167756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464301bb-2132-4161-8b18-0bf70f9b6631 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:18.167875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464301bb-2132-4161-8b18-0bf70f9b6631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:18.243169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-464301bb-2132-4161-8b18-0bf70f9b6631 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:18.244364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-464301bb-2132-4161-8b18-0bf70f9b6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:18.246377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:18.363351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:18.519458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cefab5-fc7f-43df-be33-f80c9dc45d3a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:18.557807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c0cefab5-fc7f-43df-be33-f80c9dc45d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:18.682030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cefab5-fc7f-43df-be33-f80c9dc45d3a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:18.710691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:18.797008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54c097-7aac-4c00-9721-9598af1ca39e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:18.797644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54c097-7aac-4c00-9721-9598af1ca39e service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:18.797923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54c097-7aac-4c00-9721-9598af1ca39e service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:18.798191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54c097-7aac-4c00-9721-9598af1ca39e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:18.799831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bd54c097-7aac-4c00-9721-9598af1ca39e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:18.888645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54c097-7aac-4c00-9721-9598af1ca39e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:18.890243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54c097-7aac-4c00-9721-9598af1ca39e service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:18.907921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 711/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:20.716592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58dda-204a-4925-8985-7e44fd34d48c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:20.717155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58dda-204a-4925-8985-7e44fd34d48c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:20.717155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58dda-204a-4925-8985-7e44fd34d48c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:20.718491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:20.723883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b8fac-9bfc-4539-853f-3e832395a211 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:20.724265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b8fac-9bfc-4539-853f-3e832395a211 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:20.724424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b8fac-9bfc-4539-853f-3e832395a211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:20.765376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:20.792202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1e7b8fac-9bfc-4539-853f-3e832395a211 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:20.793411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1e7b8fac-9bfc-4539-853f-3e832395a211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:20.889202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:21.172262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58dda-204a-4925-8985-7e44fd34d48c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:21.173808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2bd58dda-204a-4925-8985-7e44fd34d48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:21.276041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58dda-204a-4925-8985-7e44fd34d48c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:21.308055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:21.349243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c19761-e147-4f13-bc5f-1d669a77006f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:21.349878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c19761-e147-4f13-bc5f-1d669a77006f service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:21.350160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c19761-e147-4f13-bc5f-1d669a77006f service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:21.350433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c19761-e147-4f13-bc5f-1d669a77006f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:21.352090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e0c19761-e147-4f13-bc5f-1d669a77006f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:21.461073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c19761-e147-4f13-bc5f-1d669a77006f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:21.462434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c19761-e147-4f13-bc5f-1d669a77006f service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:21.481078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 731/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:23.531561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eb065-2502-4b67-a05d-1a01434c1f07 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:23.532113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eb065-2502-4b67-a05d-1a01434c1f07 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:23.532113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eb065-2502-4b67-a05d-1a01434c1f07 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:23.533510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:23.539011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e40e-5ae8-46c1-a162-cf3a10f3c613 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:23.540712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e40e-5ae8-46c1-a162-cf3a10f3c613 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:23.540712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e40e-5ae8-46c1-a162-cf3a10f3c613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:23.605950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9787e40e-5ae8-46c1-a162-cf3a10f3c613 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:23.607706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9787e40e-5ae8-46c1-a162-cf3a10f3c613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:23.620579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:23.707824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:23.989742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eb065-2502-4b67-a05d-1a01434c1f07 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:23.991353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6b3eb065-2502-4b67-a05d-1a01434c1f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:24.093829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eb065-2502-4b67-a05d-1a01434c1f07 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:24.106086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:24.168878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753234f-b494-4dd1-b7cb-e9e9eb2aa0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:24.169482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753234f-b494-4dd1-b7cb-e9e9eb2aa0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:24.169763 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753234f-b494-4dd1-b7cb-e9e9eb2aa0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:24.170037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753234f-b494-4dd1-b7cb-e9e9eb2aa0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:24.171668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1753234f-b494-4dd1-b7cb-e9e9eb2aa0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:24.279678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753234f-b494-4dd1-b7cb-e9e9eb2aa0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:24.281106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753234f-b494-4dd1-b7cb-e9e9eb2aa0a0 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:24.298733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 714/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:26.542707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982be80-709c-406e-acae-1d89f0822c1c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:26.543297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982be80-709c-406e-acae-1d89f0822c1c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:26.543297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982be80-709c-406e-acae-1d89f0822c1c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:26.545145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:26.550456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381b523-5390-4b9b-a9fa-a70b6454ffe5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:26.550858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381b523-5390-4b9b-a9fa-a70b6454ffe5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:26.551026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381b523-5390-4b9b-a9fa-a70b6454ffe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:26.599400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:26.617383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1381b523-5390-4b9b-a9fa-a70b6454ffe5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:26.618582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1381b523-5390-4b9b-a9fa-a70b6454ffe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:26.714784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:26.996657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982be80-709c-406e-acae-1d89f0822c1c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:26.998813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e982be80-709c-406e-acae-1d89f0822c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:27.101880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982be80-709c-406e-acae-1d89f0822c1c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:27.141547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:27.174673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf3ba8-7291-4a9e-a4d7-3b670db32dc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:27.175205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf3ba8-7291-4a9e-a4d7-3b670db32dc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:27.175482 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf3ba8-7291-4a9e-a4d7-3b670db32dc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:27.175751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf3ba8-7291-4a9e-a4d7-3b670db32dc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:27.177341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3ebf3ba8-7291-4a9e-a4d7-3b670db32dc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:27.284727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf3ba8-7291-4a9e-a4d7-3b670db32dc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:27.286077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf3ba8-7291-4a9e-a4d7-3b670db32dc6 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:27.305580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 734/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:29.366600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094abe09-ccae-4733-9303-36c8f0a909a1 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:29.367155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094abe09-ccae-4733-9303-36c8f0a909a1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:29.367155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094abe09-ccae-4733-9303-36c8f0a909a1 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:29.368515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:29.373747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1fb6-144e-490b-a35f-7e6d6eaf0155 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:29.374194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1fb6-144e-490b-a35f-7e6d6eaf0155 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:29.374301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada1fb6-144e-490b-a35f-7e6d6eaf0155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:29.439561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5ada1fb6-144e-490b-a35f-7e6d6eaf0155 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:29.440702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5ada1fb6-144e-490b-a35f-7e6d6eaf0155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:29.461651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:29.537651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:29.821021 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094abe09-ccae-4733-9303-36c8f0a909a1 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:29.822868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-094abe09-ccae-4733-9303-36c8f0a909a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:29.926851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094abe09-ccae-4733-9303-36c8f0a909a1 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:29.951583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:30.004760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cba7c-404c-426a-ab90-005b498dac4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:30.005360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cba7c-404c-426a-ab90-005b498dac4e service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:30.005635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cba7c-404c-426a-ab90-005b498dac4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:30.005906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cba7c-404c-426a-ab90-005b498dac4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:30.007527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e86cba7c-404c-426a-ab90-005b498dac4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:30.114412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cba7c-404c-426a-ab90-005b498dac4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:30.115793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cba7c-404c-426a-ab90-005b498dac4e service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:30.133646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 717/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:32.057443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931d0b-7c27-4a89-af91-94f71e485bad None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:32.058026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931d0b-7c27-4a89-af91-94f71e485bad None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:32.058026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931d0b-7c27-4a89-af91-94f71e485bad None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:32.059314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:32.064593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef286b-72bf-4c3e-a109-22646c71c5df None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:32.065029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef286b-72bf-4c3e-a109-22646c71c5df None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:32.065130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef286b-72bf-4c3e-a109-22646c71c5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:32.128758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:32.131451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-45ef286b-72bf-4c3e-a109-22646c71c5df None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:32.132653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-45ef286b-72bf-4c3e-a109-22646c71c5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:32.230141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:32.516279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74931d0b-7c27-4a89-af91-94f71e485bad None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:32.517955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-74931d0b-7c27-4a89-af91-94f71e485bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:32.618710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74931d0b-7c27-4a89-af91-94f71e485bad None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:32.666308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:32.690975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd5804-2a0c-40e5-bffe-3df295df9d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:32.691571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd5804-2a0c-40e5-bffe-3df295df9d9c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:32.691849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd5804-2a0c-40e5-bffe-3df295df9d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:32.692119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd5804-2a0c-40e5-bffe-3df295df9d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:32.693734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5dcd5804-2a0c-40e5-bffe-3df295df9d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:32.802152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd5804-2a0c-40e5-bffe-3df295df9d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:32.803622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd5804-2a0c-40e5-bffe-3df295df9d9c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:32.822217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 737/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:34.760467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bcc16-dc5c-466b-b9c2-e9863854f510 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:34.761058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bcc16-dc5c-466b-b9c2-e9863854f510 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:34.761058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bcc16-dc5c-466b-b9c2-e9863854f510 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:34.762438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:34.767780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17aca04-4880-443a-b3fe-761a572f15aa None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:34.768200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17aca04-4880-443a-b3fe-761a572f15aa None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:34.768331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17aca04-4880-443a-b3fe-761a572f15aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:34.952531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d17aca04-4880-443a-b3fe-761a572f15aa None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:34.953783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d17aca04-4880-443a-b3fe-761a572f15aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:34.986415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:35.054527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:35.339371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bcc16-dc5c-466b-b9c2-e9863854f510 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:35.341425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-170bcc16-dc5c-466b-b9c2-e9863854f510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:35.442840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bcc16-dc5c-466b-b9c2-e9863854f510 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:35.471060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:35.515578 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee749e-1a0f-4557-88a6-27e9890212f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:35.516183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee749e-1a0f-4557-88a6-27e9890212f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:35.516458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee749e-1a0f-4557-88a6-27e9890212f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:35.516727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee749e-1a0f-4557-88a6-27e9890212f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:35.518409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b4ee749e-1a0f-4557-88a6-27e9890212f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:35.626798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee749e-1a0f-4557-88a6-27e9890212f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:35.628264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee749e-1a0f-4557-88a6-27e9890212f2 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:35.647067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 720/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:37.584665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff329340-45a8-446a-b29f-0b1c70c6f1ab None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:37.585180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff329340-45a8-446a-b29f-0b1c70c6f1ab None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:37.585180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff329340-45a8-446a-b29f-0b1c70c6f1ab None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:37.586491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:37.592284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a3793-e159-4984-a15c-b598b05195c2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:37.592733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a3793-e159-4984-a15c-b598b05195c2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:37.592844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a3793-e159-4984-a15c-b598b05195c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:37.658827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a57a3793-e159-4984-a15c-b598b05195c2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:37.660060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a57a3793-e159-4984-a15c-b598b05195c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:37.668431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:37.758374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:38.039933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff329340-45a8-446a-b29f-0b1c70c6f1ab None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:38.041507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ff329340-45a8-446a-b29f-0b1c70c6f1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:38.142909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff329340-45a8-446a-b29f-0b1c70c6f1ab None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:38.196751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:38.215249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4ffbb-50f6-4cd7-bf53-314571d5c4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:38.215851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ffbb-50f6-4cd7-bf53-314571d5c4eb service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:38.216131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ffbb-50f6-4cd7-bf53-314571d5c4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:38.216401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ffbb-50f6-4cd7-bf53-314571d5c4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:38.218011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-56e4ffbb-50f6-4cd7-bf53-314571d5c4eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:38.324931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4ffbb-50f6-4cd7-bf53-314571d5c4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:38.326316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4ffbb-50f6-4cd7-bf53-314571d5c4eb service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:38.343854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 740/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:40.289581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96110867-a370-49b9-acea-0ff271b322c2 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:40.290164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96110867-a370-49b9-acea-0ff271b322c2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:40.290164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96110867-a370-49b9-acea-0ff271b322c2 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:40.291521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:40.296883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158485f-0f3d-491a-a58b-1bbbc11e69fc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:40.297324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158485f-0f3d-491a-a58b-1bbbc11e69fc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:40.297432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158485f-0f3d-491a-a58b-1bbbc11e69fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:40.363940 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3158485f-0f3d-491a-a58b-1bbbc11e69fc None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:40.365152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3158485f-0f3d-491a-a58b-1bbbc11e69fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:40.403343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:40.461184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:40.747227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96110867-a370-49b9-acea-0ff271b322c2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:40.748803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-96110867-a370-49b9-acea-0ff271b322c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:40.852960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96110867-a370-49b9-acea-0ff271b322c2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:40.891758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:40.926924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b06bca-e717-40d7-a13b-ee16d47f6470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:40.927544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b06bca-e717-40d7-a13b-ee16d47f6470 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:40.927823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b06bca-e717-40d7-a13b-ee16d47f6470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:40.928097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b06bca-e717-40d7-a13b-ee16d47f6470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:40.929696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c3b06bca-e717-40d7-a13b-ee16d47f6470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:41.038279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b06bca-e717-40d7-a13b-ee16d47f6470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:41.039679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b06bca-e717-40d7-a13b-ee16d47f6470 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:41.058114 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 723/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:43.008991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fc145-e504-4ece-a142-eac3b8b96585 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:43.009553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fc145-e504-4ece-a142-eac3b8b96585 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:43.009553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fc145-e504-4ece-a142-eac3b8b96585 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:43.010975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:43.016793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1123-de38-407a-9be8-57409aa9beda None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:43.017297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1123-de38-407a-9be8-57409aa9beda None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:43.017426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1123-de38-407a-9be8-57409aa9beda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:43.084359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b35d1123-de38-407a-9be8-57409aa9beda None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:43.085528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b35d1123-de38-407a-9be8-57409aa9beda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:43.103252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:43.181980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:43.465935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fc145-e504-4ece-a142-eac3b8b96585 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:43.467546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ab2fc145-e504-4ece-a142-eac3b8b96585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:43.571865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fc145-e504-4ece-a142-eac3b8b96585 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:43.637834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:43.644453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8a461-59ab-4a9f-aed3-1bf7d0bdaa30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:43.645047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a461-59ab-4a9f-aed3-1bf7d0bdaa30 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:43.645319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a461-59ab-4a9f-aed3-1bf7d0bdaa30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:43.645581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a461-59ab-4a9f-aed3-1bf7d0bdaa30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:43.647212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c7b8a461-59ab-4a9f-aed3-1bf7d0bdaa30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:43.752898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8a461-59ab-4a9f-aed3-1bf7d0bdaa30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:43.754245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8a461-59ab-4a9f-aed3-1bf7d0bdaa30 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:43.772414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 743/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:45.738747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489effb-7b96-4544-b122-fe6b4634cef4 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:45.739317 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489effb-7b96-4544-b122-fe6b4634cef4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:45.739317 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489effb-7b96-4544-b122-fe6b4634cef4 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:45.740648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:09:45.745894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2f278-561c-4c42-868f-ee4687c5e860 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:45.746384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2f278-561c-4c42-868f-ee4687c5e860 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:45.746605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2f278-561c-4c42-868f-ee4687c5e860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:45.811659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-82a2f278-561c-4c42-868f-ee4687c5e860 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:09:45.812854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-82a2f278-561c-4c42-868f-ee4687c5e860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:45.860926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:45.907231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:09:46.187161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489effb-7b96-4544-b122-fe6b4634cef4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:09:46.188716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7489effb-7b96-4544-b122-fe6b4634cef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:46.292816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489effb-7b96-4544-b122-fe6b4634cef4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:09:46.340766 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:46.365740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7548c4-ea25-4436-8292-9625a2296504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:09:46.366332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7548c4-ea25-4436-8292-9625a2296504 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:09:46.366606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7548c4-ea25-4436-8292-9625a2296504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:09:46.366872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7548c4-ea25-4436-8292-9625a2296504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:09:46.368474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ff7548c4-ea25-4436-8292-9625a2296504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:09:46.475439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7548c4-ea25-4436-8292-9625a2296504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:09:46.476814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7548c4-ea25-4436-8292-9625a2296504 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:09:46.494064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 726/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:09:55.913879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=22 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:09:56.491892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=66 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:05.853813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f4dbc-dc6f-4c96-83a7-1792b8b2370e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:05.854381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f4dbc-dc6f-4c96-83a7-1792b8b2370e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:05.854381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f4dbc-dc6f-4c96-83a7-1792b8b2370e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:05.855727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 745/1471] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:10:05.861100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a4e42-2e2f-4eb8-a32b-be1d3569aa84 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:05.861516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a4e42-2e2f-4eb8-a32b-be1d3569aa84 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:05.861689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a4e42-2e2f-4eb8-a32b-be1d3569aa84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:05.927613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-de7a4e42-2e2f-4eb8-a32b-be1d3569aa84 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:10:05.928827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-de7a4e42-2e2f-4eb8-a32b-be1d3569aa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:06.024995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 727/1472] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:10:06.324661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b856e0-2aeb-47c9-9854-1ff440df8bbe None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:06.324945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b856e0-2aeb-47c9-9854-1ff440df8bbe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:06.325210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b856e0-2aeb-47c9-9854-1ff440df8bbe None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:06.326541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 746/1473] 10.222.0.58 () {54 vars in 967 bytes} [Tue Aug 23 08:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:10:06.331460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ecab1-3858-4f23-9109-b29ce2eca918 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:06.331748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ecab1-3858-4f23-9109-b29ce2eca918 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:06.332026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ecab1-3858-4f23-9109-b29ce2eca918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:06.410539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ea5ecab1-3858-4f23-9109-b29ce2eca918 None None] MFA Rules not processed for user `5e9526306889402c83f2d33b340c2dac`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:10:06.411783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ea5ecab1-3858-4f23-9109-b29ce2eca918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:06.432527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:06.487299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 728/1474] 10.222.0.58 () {58 vars in 1064 bytes} [Tue Aug 23 08:10:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:10:06.495509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b856e0-2aeb-47c9-9854-1ff440df8bbe None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:10:06.497031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-20b856e0-2aeb-47c9-9854-1ff440df8bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:06.599843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b856e0-2aeb-47c9-9854-1ff440df8bbe None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:10:06.672503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8a440-d5e7-47e6-9919-163801d24bf0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e9526306889402c83f2d33b340c2dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:10:06.673101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8a440-d5e7-47e6-9919-163801d24bf0 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:06.673374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8a440-d5e7-47e6-9919-163801d24bf0 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:06.673642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8a440-d5e7-47e6-9919-163801d24bf0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:06.675247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f3c8a440-d5e7-47e6-9919-163801d24bf0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:06.729593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:06.784917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8a440-d5e7-47e6-9919-163801d24bf0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:10:06.786367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8a440-d5e7-47e6-9919-163801d24bf0 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:10:06.805086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 747/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:10:08.793344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d89881-20b1-4c65-a166-f3cce7f7068c None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:08.793817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d89881-20b1-4c65-a166-f3cce7f7068c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:08.793921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d89881-20b1-4c65-a166-f3cce7f7068c None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:08.795325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 729/1476] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:10:08.800454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029722b-a3c1-492a-a49a-083c9fdc2c62 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:08.800859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029722b-a3c1-492a-a49a-083c9fdc2c62 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:08.800998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029722b-a3c1-492a-a49a-083c9fdc2c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:08.866366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c029722b-a3c1-492a-a49a-083c9fdc2c62 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:10:08.867591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c029722b-a3c1-492a-a49a-083c9fdc2c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:08.963521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 748/1477] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:10:09.235542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d89881-20b1-4c65-a166-f3cce7f7068c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:10:09.237140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-11d89881-20b1-4c65-a166-f3cce7f7068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:09.362811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d89881-20b1-4c65-a166-f3cce7f7068c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:10:09.411261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:09.455180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb4b8e-d153-4a12-8365-6b94bacb032e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:10:09.455764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4b8e-d153-4a12-8365-6b94bacb032e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:09.456057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4b8e-d153-4a12-8365-6b94bacb032e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:09.456329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb4b8e-d153-4a12-8365-6b94bacb032e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:09.456964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb4b8e-d153-4a12-8365-6b94bacb032e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:10:09.458744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-6beb4b8e-d153-4a12-8365-6b94bacb032e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:10:09.458744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Aug 23 08:10:09.458744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: Aug 23 08:10:09.460036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb4b8e-d153-4a12-8365-6b94bacb032e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:10:09.481413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 730/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Aug 23 08:10:09 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:10:11.397641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a606-e56e-4e6e-84be-8384ba1c4b25 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:11.398235 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a606-e56e-4e6e-84be-8384ba1c4b25 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:11.398387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a606-e56e-4e6e-84be-8384ba1c4b25 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:11.399668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 749/1479] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:10:11.404838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff340c23-fd6d-4384-842b-5fc12b655f22 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:11.405276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff340c23-fd6d-4384-842b-5fc12b655f22 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:11.405382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff340c23-fd6d-4384-842b-5fc12b655f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:11.470891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ff340c23-fd6d-4384-842b-5fc12b655f22 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:10:11.472073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ff340c23-fd6d-4384-842b-5fc12b655f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:11.570223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 731/1480] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:10:11.862055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5a606-e56e-4e6e-84be-8384ba1c4b25 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:10:11.863648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-21f5a606-e56e-4e6e-84be-8384ba1c4b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:11.873601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:12.008700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5a606-e56e-4e6e-84be-8384ba1c4b25 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:10:12.090035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:12.123162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d851d5-a2ca-4658-8c9f-0e8547d1d5fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:10:12.126170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d851d5-a2ca-4658-8c9f-0e8547d1d5fb service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:12.126170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d851d5-a2ca-4658-8c9f-0e8547d1d5fb service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:12.126170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d851d5-a2ca-4658-8c9f-0e8547d1d5fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:12.126170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-55d851d5-a2ca-4658-8c9f-0e8547d1d5fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:12.214564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d851d5-a2ca-4658-8c9f-0e8547d1d5fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:10:12.216205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d851d5-a2ca-4658-8c9f-0e8547d1d5fb service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:10:12.233971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 750/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:10:14.324326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775cab-578c-4d7a-9373-115213993faa None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:14.324867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775cab-578c-4d7a-9373-115213993faa None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:14.324867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775cab-578c-4d7a-9373-115213993faa None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:14.326251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 732/1482] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:10:14.331751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90354ce-1443-4ebe-804b-67108e5cfe4b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:14.332175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90354ce-1443-4ebe-804b-67108e5cfe4b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:14.332288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90354ce-1443-4ebe-804b-67108e5cfe4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:14.397584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e90354ce-1443-4ebe-804b-67108e5cfe4b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:10:14.398793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e90354ce-1443-4ebe-804b-67108e5cfe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:14.444206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:14.493594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 751/1483] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:10:14.779073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775cab-578c-4d7a-9373-115213993faa None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:10:14.780639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-22775cab-578c-4d7a-9373-115213993faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:14.804757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:14.886853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775cab-578c-4d7a-9373-115213993faa None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:10:14.961759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8f491-3e39-4272-ae48-ef4cbeb243b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:10:14.962361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f491-3e39-4272-ae48-ef4cbeb243b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:14.962675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f491-3e39-4272-ae48-ef4cbeb243b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:14.962972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f491-3e39-4272-ae48-ef4cbeb243b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:14.964581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-93f8f491-3e39-4272-ae48-ef4cbeb243b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:15.039863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:15.075213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8f491-3e39-4272-ae48-ef4cbeb243b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:10:15.076790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8f491-3e39-4272-ae48-ef4cbeb243b7 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:10:15.095590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 733/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:10:17.022891 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444bfa6-2a19-4ba2-9b7b-d671478fd241 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:17.023472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444bfa6-2a19-4ba2-9b7b-d671478fd241 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:17.023472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444bfa6-2a19-4ba2-9b7b-d671478fd241 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:17.025329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 752/1485] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:10:17.030464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf44c-96c2-4f22-a069-7dd9cb8e95f7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:17.030863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf44c-96c2-4f22-a069-7dd9cb8e95f7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:17.031036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf44c-96c2-4f22-a069-7dd9cb8e95f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:17.096986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c06cf44c-96c2-4f22-a069-7dd9cb8e95f7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:10:17.098117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c06cf44c-96c2-4f22-a069-7dd9cb8e95f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:17.193218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 734/1486] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:10:17.481436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444bfa6-2a19-4ba2-9b7b-d671478fd241 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:10:17.483060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c444bfa6-2a19-4ba2-9b7b-d671478fd241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:17.587088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444bfa6-2a19-4ba2-9b7b-d671478fd241 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:10:17.657820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:17.659607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd1300-8df5-4b83-ac56-82b093f347e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:10:17.660209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd1300-8df5-4b83-ac56-82b093f347e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:17.660484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd1300-8df5-4b83-ac56-82b093f347e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:17.660755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd1300-8df5-4b83-ac56-82b093f347e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:17.662352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-33cd1300-8df5-4b83-ac56-82b093f347e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:17.771382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd1300-8df5-4b83-ac56-82b093f347e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:10:17.772747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd1300-8df5-4b83-ac56-82b093f347e8 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:10:17.791452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 753/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:10:19.810393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db29c6-cdd7-46df-9f3a-4f718a003097 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:19.810880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db29c6-cdd7-46df-9f3a-4f718a003097 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:19.811009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db29c6-cdd7-46df-9f3a-4f718a003097 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:19.812860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 735/1488] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:10:19.818084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532399f8-dcfd-4589-a751-54c5b2725b3e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:19.818531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532399f8-dcfd-4589-a751-54c5b2725b3e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:19.818640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532399f8-dcfd-4589-a751-54c5b2725b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:19.884112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-532399f8-dcfd-4589-a751-54c5b2725b3e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:10:19.885311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-532399f8-dcfd-4589-a751-54c5b2725b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:19.936812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:19.979593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 754/1489] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:10:20.262061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db29c6-cdd7-46df-9f3a-4f718a003097 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:10:20.263688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-29db29c6-cdd7-46df-9f3a-4f718a003097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:20.299723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:20.367323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db29c6-cdd7-46df-9f3a-4f718a003097 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:10:20.439829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626c8ac-c583-47af-9403-380c89653c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:10:20.440425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626c8ac-c583-47af-9403-380c89653c67 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:20.440701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626c8ac-c583-47af-9403-380c89653c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:20.440971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626c8ac-c583-47af-9403-380c89653c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:20.442555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5626c8ac-c583-47af-9403-380c89653c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:20.524476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:20.550841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626c8ac-c583-47af-9403-380c89653c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:10:20.552354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626c8ac-c583-47af-9403-380c89653c67 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:10:20.572943 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 736/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:10:22.806869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88f2ea-00b5-44b8-821e-629235539eb3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:22.807338 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88f2ea-00b5-44b8-821e-629235539eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:22.807449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88f2ea-00b5-44b8-821e-629235539eb3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:22.808829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 755/1491] 10.222.0.58 () {54 vars in 927 bytes} [Tue Aug 23 08:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:10:22.814185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0de62-79f3-4294-afaa-b89ddad7f7fe None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:22.814602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0de62-79f3-4294-afaa-b89ddad7f7fe None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:22.814733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0de62-79f3-4294-afaa-b89ddad7f7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:22.880527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-43e0de62-79f3-4294-afaa-b89ddad7f7fe None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:10:22.881716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-43e0de62-79f3-4294-afaa-b89ddad7f7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:22.975887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 737/1492] 10.222.0.58 () {58 vars in 1024 bytes} [Tue Aug 23 08:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:10:23.268601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88f2ea-00b5-44b8-821e-629235539eb3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:10:23.270188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e88f2ea-00b5-44b8-821e-629235539eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:23.372168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88f2ea-00b5-44b8-821e-629235539eb3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:10:23.446644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284b3b3-f15c-4564-b3a4-36c09fd7f6d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:10:23.447277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284b3b3-f15c-4564-b3a4-36c09fd7f6d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:10:23.447554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284b3b3-f15c-4564-b3a4-36c09fd7f6d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:10:23.447821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284b3b3-f15c-4564-b3a4-36c09fd7f6d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:10:23.449432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a284b3b3-f15c-4564-b3a4-36c09fd7f6d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:10:23.462903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:23.559139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284b3b3-f15c-4564-b3a4-36c09fd7f6d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:10:23.560649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284b3b3-f15c-4564-b3a4-36c09fd7f6d7 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:10:23.579174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 756/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:10:32.974097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=82 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:10:33.576929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=50 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:13:27.680177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ba59-0d26-4e49-8314-2d93fc11ec01 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:27.680753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ba59-0d26-4e49-8314-2d93fc11ec01 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:27.680753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ba59-0d26-4e49-8314-2d93fc11ec01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:27.761659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d100ba59-0d26-4e49-8314-2d93fc11ec01 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:13:27.762909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d100ba59-0d26-4e49-8314-2d93fc11ec01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:13:27.866706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 738/1494] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:13:27.874904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d3b5f-7dc1-4b75-8706-b04dfb871541 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:27.875422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d3b5f-7dc1-4b75-8706-b04dfb871541 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:27.875526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d3b5f-7dc1-4b75-8706-b04dfb871541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:27.895369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-028d3b5f-7dc1-4b75-8706-b04dfb871541 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:13:27.896564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-028d3b5f-7dc1-4b75-8706-b04dfb871541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:13:27.905650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 757/1495] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:13:27.910295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100ba59-0d26-4e49-8314-2d93fc11ec01 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:27.920312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100ba59-0d26-4e49-8314-2d93fc11ec01 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:27.924915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:27.925503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:27.925778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:27.926047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:27.926686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:13:27.927028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:27.928029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:13:27.928029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Aug 23 08:13:27.928029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: Aug 23 08:13:27.928908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:27.940402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 739/1496] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:13:27.948890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d3b5f-7dc1-4b75-8706-b04dfb871541 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:27.957887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d3b5f-7dc1-4b75-8706-b04dfb871541 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:27.963022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:27.963608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:27.963883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:27.964151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:27.964758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:27.966478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:13:27.966478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 23 08:13:27.966478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: Aug 23 08:13:27.967689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:28.094063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 758/1497] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:13:27 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:13:28.098474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:28.108468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d393a-201b-49ab-bfc3-50e6fd725fd6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:28.157377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:13:28.164113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:28.164704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.164982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.165255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.165944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:28.166973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:13:28.166973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Aug 23 08:13:28.166973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: Aug 23 08:13:28.169884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:28.273459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 740/1498] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:13:28 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:13:28.277790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:28.288041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c2071-0bde-4fd4-950d-27424ef05c60 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:28.292680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:28.293520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.293790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.294053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.294680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:28.295729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:13:28.295729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 23 08:13:28.295729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: Aug 23 08:13:28.296721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:28.309692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 759/1499] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:13:28.313666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:28.324235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231eeb9-dc66-4147-abfe-62875241a40f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:28.328848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff321e56-ca61-4b9c-96da-bb8c97cc7eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:28.329435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff321e56-ca61-4b9c-96da-bb8c97cc7eae admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.329718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff321e56-ca61-4b9c-96da-bb8c97cc7eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.329993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff321e56-ca61-4b9c-96da-bb8c97cc7eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.330631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff321e56-ca61-4b9c-96da-bb8c97cc7eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:28.331682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-ff321e56-ca61-4b9c-96da-bb8c97cc7eae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:13:28.331682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Aug 23 08:13:28.331682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: Aug 23 08:13:28.332565 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff321e56-ca61-4b9c-96da-bb8c97cc7eae admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:28.345710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 741/1500] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:13:28.349683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:28.359520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6156b-12ae-44f3-af80-85d91af0e351 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:28.364114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e940069-9c90-47f0-9ad7-5ab9ce194b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:28.364749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940069-9c90-47f0-9ad7-5ab9ce194b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.365030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940069-9c90-47f0-9ad7-5ab9ce194b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.365305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940069-9c90-47f0-9ad7-5ab9ce194b5b admin admin] PATH_INFO: `/v3/projects/d56ef0635912464aaedc1c6c6a793547/users/0e777ff3cd2349fdab88dde9d070250d/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.380163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e940069-9c90-47f0-9ad7-5ab9ce194b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56ef0635912464aaedc1c6c6a793547, user_id=0e777ff3cd2349fdab88dde9d070250d, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:28.381182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-7e940069-9c90-47f0-9ad7-5ab9ce194b5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:13:28.381182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 23 08:13:28.381182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: Aug 23 08:13:28.383171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e940069-9c90-47f0-9ad7-5ab9ce194b5b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:28.450465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 760/1501] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:13:28 2022] PUT /identity/v3/projects/d56ef0635912464aaedc1c6c6a793547/users/0e777ff3cd2349fdab88dde9d070250d/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:13:28.462319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac239-58aa-4803-85e7-fecdf6646328 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.462636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac239-58aa-4803-85e7-fecdf6646328 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.462909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac239-58aa-4803-85e7-fecdf6646328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.491064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-7e3ac239-58aa-4803-85e7-fecdf6646328 None None] MFA Rules not processed for user `0e777ff3cd2349fdab88dde9d070250d`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:13:28.494303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7e3ac239-58aa-4803-85e7-fecdf6646328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:13:28.674283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 742/1502] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:13:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:13:28.688368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185d280-64d2-45c6-85a9-3c785729fbf1 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.688654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185d280-64d2-45c6-85a9-3c785729fbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.688929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185d280-64d2-45c6-85a9-3c785729fbf1 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.690287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 761/1503] 10.222.0.58 () {54 vars in 964 bytes} [Tue Aug 23 08:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:13:28.695077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c41bc-7db6-4932-96e1-12e575c2a9be None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.695530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c41bc-7db6-4932-96e1-12e575c2a9be None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.695638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c41bc-7db6-4932-96e1-12e575c2a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.753080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-553c41bc-7db6-4932-96e1-12e575c2a9be None None] MFA Rules not processed for user `6adffd03ce3a4ebfb2d7c5df3a2ca293`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:13:28.754271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-553c41bc-7db6-4932-96e1-12e575c2a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:13:28.828615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 743/1504] 10.222.0.58 () {58 vars in 1061 bytes} [Tue Aug 23 08:13:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:13:28.836451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185d280-64d2-45c6-85a9-3c785729fbf1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:28.846665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185d280-64d2-45c6-85a9-3c785729fbf1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:28.850072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f57a5-86e0-4971-ab09-5b279f0cb45f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:28.850649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f57a5-86e0-4971-ab09-5b279f0cb45f service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.850921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f57a5-86e0-4971-ab09-5b279f0cb45f service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.851223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f57a5-86e0-4971-ab09-5b279f0cb45f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.859522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f57a5-86e0-4971-ab09-5b279f0cb45f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:28.860940 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f57a5-86e0-4971-ab09-5b279f0cb45f service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:28.870033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 762/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:13:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:13:28.913599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64244cc6-b9ab-48b7-83a9-ce17ec7c407a None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.913885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64244cc6-b9ab-48b7-83a9-ce17ec7c407a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.914154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64244cc6-b9ab-48b7-83a9-ce17ec7c407a None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.915948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 744/1506] 10.222.0.58 () {52 vars in 830 bytes} [Tue Aug 23 08:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:13:28.919696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f57a5-86e0-4971-ab09-5b279f0cb45f service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:28.980434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f57a5-86e0-4971-ab09-5b279f0cb45f service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:28.984838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:28.985451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:28.985725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:28.985995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] PATH_INFO: `/v3/users/0e777ff3cd2349fdab88dde9d070250d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:28.987132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e777ff3cd2349fdab88dde9d070250d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:28.988137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:13:28.988137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 23 08:13:28.988137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: Aug 23 08:13:28.989329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:29.270744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e777ff3cd2349fdab88dde9d070250d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:29.285045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e777ff3cd2349fdab88dde9d070250d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:29.285834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e777ff3cd2349fdab88dde9d070250d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:29.286448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e777ff3cd2349fdab88dde9d070250d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:29.312176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e777ff3cd2349fdab88dde9d070250d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:29.316209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:13:29.393220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 763/1507] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:13:28 2022] DELETE /identity/v3/users/0e777ff3cd2349fdab88dde9d070250d => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:13:29.397548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52fa54-7f57-4551-bb34-c5010f225a74 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:29.397904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52fa54-7f57-4551-bb34-c5010f225a74 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:29.398184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52fa54-7f57-4551-bb34-c5010f225a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:29.418619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fd52fa54-7f57-4551-bb34-c5010f225a74 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:13:29.419837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fd52fa54-7f57-4551-bb34-c5010f225a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:13:29.429111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:13:29.479080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 745/1508] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:13:29.489354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:29.490828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:13:29.567246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890d6ba-9f04-41bd-b034-d6481f640cbb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:29.570727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd7ef7-482c-4155-80f7-bd395fcf6f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:29.571354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7ef7-482c-4155-80f7-bd395fcf6f35 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:29.572220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7ef7-482c-4155-80f7-bd395fcf6f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:29.572688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7ef7-482c-4155-80f7-bd395fcf6f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:29.582244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd7ef7-482c-4155-80f7-bd395fcf6f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:29.583703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd7ef7-482c-4155-80f7-bd395fcf6f35 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:29.593492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 764/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:13:30.047088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52fa54-7f57-4551-bb34-c5010f225a74 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:13:30.050461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fd52fa54-7f57-4551-bb34-c5010f225a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:13:30.061246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52fa54-7f57-4551-bb34-c5010f225a74 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:13:30.065935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:13:30.066555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:13:30.066831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:13:30.067125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] PATH_INFO: `/v3/projects/d56ef0635912464aaedc1c6c6a793547` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:13:30.068336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56ef0635912464aaedc1c6c6a793547)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:13:30.069342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:13:30.069342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Aug 23 08:13:30.069342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: Aug 23 08:13:30.070530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:13:30.198549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56ef0635912464aaedc1c6c6a793547', 'request_id': 'req-b33933f8-862f-4ac5-a338-53d9505a0cae'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:30.204569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56ef0635912464aaedc1c6c6a793547', 'request_id': 'req-b33933f8-862f-4ac5-a338-53d9505a0cae'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:30.217414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56ef0635912464aaedc1c6c6a793547', 'request_id': 'req-b33933f8-862f-4ac5-a338-53d9505a0cae'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:30.320220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] The token cache is being invalidate because project d56ef0635912464aaedc1c6c6a793547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:13:30.320508 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:30.321130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b33933f8-862f-4ac5-a338-53d9505a0cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:13:30.322715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 746/1510] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:13:30 2022] DELETE /identity/v3/projects/d56ef0635912464aaedc1c6c6a793547 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:13:39.589470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:13:40.315378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:15:00.166819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ffc89-4993-47dc-97e0-6d183576efa2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:00.167418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ffc89-4993-47dc-97e0-6d183576efa2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:00.167418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ffc89-4993-47dc-97e0-6d183576efa2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:00.168734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:15:00.173728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a36c-3b65-4047-99dd-0df5138cb0f8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:00.174163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a36c-3b65-4047-99dd-0df5138cb0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:00.174261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a36c-3b65-4047-99dd-0df5138cb0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:00.193882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2cc3a36c-3b65-4047-99dd-0df5138cb0f8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:00.195132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2cc3a36c-3b65-4047-99dd-0df5138cb0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:00.252968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:00.406600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ffc89-4993-47dc-97e0-6d183576efa2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:00.408431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d76ffc89-4993-47dc-97e0-6d183576efa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:00.512257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ffc89-4993-47dc-97e0-6d183576efa2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:00.518339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b844a3-3c64-485b-a154-ca952f16ef87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:00.519247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b844a3-3c64-485b-a154-ca952f16ef87 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:00.519363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b844a3-3c64-485b-a154-ca952f16ef87 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:00.519552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b844a3-3c64-485b-a154-ca952f16ef87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:00.527704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b844a3-3c64-485b-a154-ca952f16ef87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:00.529284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b844a3-3c64-485b-a154-ca952f16ef87 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:00.538915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 766/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:01.361452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab1b5b-5ad8-48cc-9bb6-a40737a6d861 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:01.361912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab1b5b-5ad8-48cc-9bb6-a40737a6d861 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:01.362020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab1b5b-5ad8-48cc-9bb6-a40737a6d861 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:01.363427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:15:01.368140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871967a9-bb2b-42a8-aa70-64aefb4f6807 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:01.368557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871967a9-bb2b-42a8-aa70-64aefb4f6807 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:01.368684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871967a9-bb2b-42a8-aa70-64aefb4f6807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:01.388031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-871967a9-bb2b-42a8-aa70-64aefb4f6807 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:01.399579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-871967a9-bb2b-42a8-aa70-64aefb4f6807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:01.399579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:01.549438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab1b5b-5ad8-48cc-9bb6-a40737a6d861 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:01.564654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab1b5b-5ad8-48cc-9bb6-a40737a6d861 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:01.570131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dd706-4e47-4681-8df1-d2ec8b47c2bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:01.570720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd706-4e47-4681-8df1-d2ec8b47c2bf service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:01.571033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd706-4e47-4681-8df1-d2ec8b47c2bf service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:01.571309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd706-4e47-4681-8df1-d2ec8b47c2bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:01.579217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dd706-4e47-4681-8df1-d2ec8b47c2bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:01.580791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dd706-4e47-4681-8df1-d2ec8b47c2bf service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:01.589993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 749/1516] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:01.953661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b0e5-b84e-4acc-a8c3-3fb56abdbc15 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:01.953949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b0e5-b84e-4acc-a8c3-3fb56abdbc15 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:01.954227 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b0e5-b84e-4acc-a8c3-3fb56abdbc15 None None] PATH_INFO: `/v3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:01.955577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 768/1517] 10.222.0.59 () {54 vars in 925 bytes} [Tue Aug 23 08:15:01 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:15:01.963315 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d25a2-af5c-45a7-9dee-21752c1d36c8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:01.963603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d25a2-af5c-45a7-9dee-21752c1d36c8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:01.963882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d25a2-af5c-45a7-9dee-21752c1d36c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:02.013568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-578d25a2-af5c-45a7-9dee-21752c1d36c8 None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:02.014779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-578d25a2-af5c-45a7-9dee-21752c1d36c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:02.085244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 750/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Tue Aug 23 08:15:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:11.210487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc89623-518e-429f-ac68-199f71c74f02 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:11.211019 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc89623-518e-429f-ac68-199f71c74f02 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:11.211135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc89623-518e-429f-ac68-199f71c74f02 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:11.212451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 769/1519] 10.222.0.58 () {54 vars in 961 bytes} [Tue Aug 23 08:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:15:11.217472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376005be-1f4d-4d46-86f9-b2bf40227704 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:11.217918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376005be-1f4d-4d46-86f9-b2bf40227704 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:11.218025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376005be-1f4d-4d46-86f9-b2bf40227704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:11.238631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-376005be-1f4d-4d46-86f9-b2bf40227704 None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:11.239890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-376005be-1f4d-4d46-86f9-b2bf40227704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:11.248558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 751/1520] 10.222.0.58 () {58 vars in 1058 bytes} [Tue Aug 23 08:15:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:11.256772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc89623-518e-429f-ac68-199f71c74f02 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:11.270327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc89623-518e-429f-ac68-199f71c74f02 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:11.275462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe3078-2c42-49b8-a732-6eb8663617df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:11.276125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3078-2c42-49b8-a732-6eb8663617df service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:11.276404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3078-2c42-49b8-a732-6eb8663617df service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:11.276675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3078-2c42-49b8-a732-6eb8663617df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:11.285438 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe3078-2c42-49b8-a732-6eb8663617df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:11.287121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe3078-2c42-49b8-a732-6eb8663617df service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:11.296878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 770/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:12.817930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852e2f-2d91-4e2e-a42d-012b088bf104 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:12.818381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852e2f-2d91-4e2e-a42d-012b088bf104 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:12.818501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852e2f-2d91-4e2e-a42d-012b088bf104 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:12.819853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:15:12.824743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cd711-ba0f-4ceb-8378-27a7faa17c25 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:12.825185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cd711-ba0f-4ceb-8378-27a7faa17c25 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:12.825282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cd711-ba0f-4ceb-8378-27a7faa17c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:12.844307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7e5cd711-ba0f-4ceb-8378-27a7faa17c25 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:12.845562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7e5cd711-ba0f-4ceb-8378-27a7faa17c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:12.853442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:13.006787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852e2f-2d91-4e2e-a42d-012b088bf104 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:13.021749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852e2f-2d91-4e2e-a42d-012b088bf104 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:13.032169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054816e-f561-4177-b921-a33b81542810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:13.032721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054816e-f561-4177-b921-a33b81542810 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:13.032998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054816e-f561-4177-b921-a33b81542810 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:13.033302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054816e-f561-4177-b921-a33b81542810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:13.042727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054816e-f561-4177-b921-a33b81542810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:13.044359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054816e-f561-4177-b921-a33b81542810 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:13.054518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 753/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:13.876550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc72118-08dc-4970-85cb-1870e0922c16 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:13.877091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc72118-08dc-4970-85cb-1870e0922c16 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:13.877091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc72118-08dc-4970-85cb-1870e0922c16 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:13.878421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 08:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 08:15:13.883637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57f455-9a50-435e-a9cf-857ea8f5f37e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:13.884033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57f455-9a50-435e-a9cf-857ea8f5f37e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:13.884192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57f455-9a50-435e-a9cf-857ea8f5f37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:13.903334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6a57f455-9a50-435e-a9cf-857ea8f5f37e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:13.904527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6a57f455-9a50-435e-a9cf-857ea8f5f37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:13.912967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 08:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:14.065794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc72118-08dc-4970-85cb-1870e0922c16 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:14.080704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc72118-08dc-4970-85cb-1870e0922c16 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:14.086180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d366e-58f9-4419-bee7-53d459fc0886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:14.086765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d366e-58f9-4419-bee7-53d459fc0886 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:14.087154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d366e-58f9-4419-bee7-53d459fc0886 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:14.087420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d366e-58f9-4419-bee7-53d459fc0886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:14.096161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d366e-58f9-4419-bee7-53d459fc0886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:14.097745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d366e-58f9-4419-bee7-53d459fc0886 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:14.106895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 773/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:17.935059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e319c-d287-4d45-bf0e-0fb47b04155d None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:17.935627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e319c-d287-4d45-bf0e-0fb47b04155d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:17.935627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e319c-d287-4d45-bf0e-0fb47b04155d None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:17.936944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:15:17.942122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e12c1-d82f-4f26-9332-fac2d293c2d7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:17.942584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e12c1-d82f-4f26-9332-fac2d293c2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:17.942692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e12c1-d82f-4f26-9332-fac2d293c2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:17.962045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-787e12c1-d82f-4f26-9332-fac2d293c2d7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:17.963292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-787e12c1-d82f-4f26-9332-fac2d293c2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:17.971653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:18.272131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e319c-d287-4d45-bf0e-0fb47b04155d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:18.273858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5c6e319c-d287-4d45-bf0e-0fb47b04155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:18.302568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:15:18.332381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e319c-d287-4d45-bf0e-0fb47b04155d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:18.335924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7e0d4-e695-4897-a9a4-316d296a9ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:18.336523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e0d4-e695-4897-a9a4-316d296a9ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:18.336805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e0d4-e695-4897-a9a4-316d296a9ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:18.337074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e0d4-e695-4897-a9a4-316d296a9ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:18.346459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e0d4-e695-4897-a9a4-316d296a9ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:18.347915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e0d4-e695-4897-a9a4-316d296a9ea6 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:18.357191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 756/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:20.221327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1d8d-df97-438b-858c-65dcdb4441f3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:20.221829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1d8d-df97-438b-858c-65dcdb4441f3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:20.221941 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1d8d-df97-438b-858c-65dcdb4441f3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:20.223290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:15:20.228610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea54b9-c3e9-40bc-a549-c6c6d50425ee None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:20.229032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea54b9-c3e9-40bc-a549-c6c6d50425ee None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:20.231874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea54b9-c3e9-40bc-a549-c6c6d50425ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:20.249193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-63ea54b9-c3e9-40bc-a549-c6c6d50425ee None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:20.250389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-63ea54b9-c3e9-40bc-a549-c6c6d50425ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:20.258233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:20.544898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a1d8d-df97-438b-858c-65dcdb4441f3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:20.555579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a1d8d-df97-438b-858c-65dcdb4441f3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:20.559485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1def5-aa1f-423a-bc8e-7e16620c0468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:20.560079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1def5-aa1f-423a-bc8e-7e16620c0468 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:20.560356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1def5-aa1f-423a-bc8e-7e16620c0468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:20.560624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1def5-aa1f-423a-bc8e-7e16620c0468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:20.569435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1def5-aa1f-423a-bc8e-7e16620c0468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:20.570859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1def5-aa1f-423a-bc8e-7e16620c0468 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:20.580112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 776/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:24.082298 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ef45-764b-4168-8081-c2e47fa3458b None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:24.082828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ef45-764b-4168-8081-c2e47fa3458b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:24.082828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ef45-764b-4168-8081-c2e47fa3458b None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:24.084206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:15:24.089569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb126876-4b52-47ee-898b-26d88a18295e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:24.090002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb126876-4b52-47ee-898b-26d88a18295e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:24.090116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb126876-4b52-47ee-898b-26d88a18295e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:24.109985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cb126876-4b52-47ee-898b-26d88a18295e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:24.111219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cb126876-4b52-47ee-898b-26d88a18295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:24.119160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:24.675277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75ef45-764b-4168-8081-c2e47fa3458b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:24.676981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7a75ef45-764b-4168-8081-c2e47fa3458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:24.732786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75ef45-764b-4168-8081-c2e47fa3458b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:24.736264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877636-c25c-47ab-9c4a-45527ffa41e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:24.736855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877636-c25c-47ab-9c4a-45527ffa41e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:24.737702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877636-c25c-47ab-9c4a-45527ffa41e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:24.737971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877636-c25c-47ab-9c4a-45527ffa41e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:24.746589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16877636-c25c-47ab-9c4a-45527ffa41e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:24.748055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16877636-c25c-47ab-9c4a-45527ffa41e8 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:24.757427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 759/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:26.649858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c225ed-fd1f-4a2e-89c2-4502beb6efec None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:26.650449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c225ed-fd1f-4a2e-89c2-4502beb6efec None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:26.650449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c225ed-fd1f-4a2e-89c2-4502beb6efec None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:26.651775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 08:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:15:26.656966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5041e2-4138-4971-b710-f76c76c1a927 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:26.657416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5041e2-4138-4971-b710-f76c76c1a927 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:26.657521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5041e2-4138-4971-b710-f76c76c1a927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:26.681546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6b5041e2-4138-4971-b710-f76c76c1a927 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:15:26.683061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6b5041e2-4138-4971-b710-f76c76c1a927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:26.690868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 08:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:15:27.243417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c225ed-fd1f-4a2e-89c2-4502beb6efec None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:15:27.245087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b4c225ed-fd1f-4a2e-89c2-4502beb6efec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:15:27.257848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c225ed-fd1f-4a2e-89c2-4502beb6efec None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:15:27.261340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50b415-e527-45ec-a4a2-8dca1eda8759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:15:27.262198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b415-e527-45ec-a4a2-8dca1eda8759 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:15:27.262474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b415-e527-45ec-a4a2-8dca1eda8759 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:15:27.262744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b415-e527-45ec-a4a2-8dca1eda8759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:15:27.271979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50b415-e527-45ec-a4a2-8dca1eda8759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:15:27.273385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50b415-e527-45ec-a4a2-8dca1eda8759 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:15:27.282569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 779/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:15:36.676292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=47 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:15:37.278511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=82 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:52.629729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b82ae8-a65f-4670-ae4a-daf2a16f3d71 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.630394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b82ae8-a65f-4670-ae4a-daf2a16f3d71 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.630394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b82ae8-a65f-4670-ae4a-daf2a16f3d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.655239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-54b82ae8-a65f-4670-ae4a-daf2a16f3d71 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:52.656940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-54b82ae8-a65f-4670-ae4a-daf2a16f3d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:52.671008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 761/1540] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:52.681032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632e0f5-ebc6-4a94-9460-0f6693bd995d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.681628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632e0f5-ebc6-4a94-9460-0f6693bd995d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.681628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632e0f5-ebc6-4a94-9460-0f6693bd995d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.691071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbbc5d-df5c-4d6e-83d1-cdd594f64a35 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.691747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbbc5d-df5c-4d6e-83d1-cdd594f64a35 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.692370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbbc5d-df5c-4d6e-83d1-cdd594f64a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.720216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a632e0f5-ebc6-4a94-9460-0f6693bd995d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:52.721531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a632e0f5-ebc6-4a94-9460-0f6693bd995d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:52.724219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0ffbbc5d-df5c-4d6e-83d1-cdd594f64a35 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:52.725483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0ffbbc5d-df5c-4d6e-83d1-cdd594f64a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:52.737769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 780/1541] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:16:52.739124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb22b12-3912-4ad8-b4ef-f217706c37ff None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.739415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb22b12-3912-4ad8-b4ef-f217706c37ff None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.739731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb22b12-3912-4ad8-b4ef-f217706c37ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.742904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 762/1542] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:16:52.743907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbbc5d-df5c-4d6e-83d1-cdd594f64a35 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:52.773595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbbc5d-df5c-4d6e-83d1-cdd594f64a35 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:52.778982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2eb22b12-3912-4ad8-b4ef-f217706c37ff None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:52.780294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2eb22b12-3912-4ad8-b4ef-f217706c37ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:52.793139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92910d2f-b71b-4b51-b475-9f6fbdf6f345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:52.793781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92910d2f-b71b-4b51-b475-9f6fbdf6f345 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.794063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92910d2f-b71b-4b51-b475-9f6fbdf6f345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.794339 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92910d2f-b71b-4b51-b475-9f6fbdf6f345 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.795048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92910d2f-b71b-4b51-b475-9f6fbdf6f345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:16:52.795355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92910d2f-b71b-4b51-b475-9f6fbdf6f345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:52.797521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92910d2f-b71b-4b51-b475-9f6fbdf6f345 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:52.806189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 781/1543] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:52.807548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9db5-0462-4153-a130-483ee6cef785 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.808430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9db5-0462-4153-a130-483ee6cef785 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.808713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9db5-0462-4153-a130-483ee6cef785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.825274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 763/1544] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:16:52.826725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9582b0-3126-4c07-8024-77a27e208730 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.827042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9582b0-3126-4c07-8024-77a27e208730 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.827323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9582b0-3126-4c07-8024-77a27e208730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.851567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-112f9db5-0462-4153-a130-483ee6cef785 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:52.852767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-112f9db5-0462-4153-a130-483ee6cef785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:52.872060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 782/1545] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:52.872952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f9db5-0462-4153-a130-483ee6cef785 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:52.877066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3d9582b0-3126-4c07-8024-77a27e208730 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:52.878297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3d9582b0-3126-4c07-8024-77a27e208730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:52.893618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f9db5-0462-4153-a130-483ee6cef785 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:52.903298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a94f07-489a-4c58-ad07-c71eb59e3537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:52.903911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a94f07-489a-4c58-ad07-c71eb59e3537 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.904188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a94f07-489a-4c58-ad07-c71eb59e3537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.904455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a94f07-489a-4c58-ad07-c71eb59e3537 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.905075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a94f07-489a-4c58-ad07-c71eb59e3537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:52.907539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a94f07-489a-4c58-ad07-c71eb59e3537 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:52.908494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 764/1546] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:52.909630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9582b0-3126-4c07-8024-77a27e208730 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:52.926933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9582b0-3126-4c07-8024-77a27e208730 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:52.944413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d90f1b-541c-4ae7-8c18-4203dac08cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:52.945293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d90f1b-541c-4ae7-8c18-4203dac08cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.945570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d90f1b-541c-4ae7-8c18-4203dac08cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.945836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d90f1b-541c-4ae7-8c18-4203dac08cb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.946497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d90f1b-541c-4ae7-8c18-4203dac08cb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:16:52.946901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d90f1b-541c-4ae7-8c18-4203dac08cb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:52.948545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d90f1b-541c-4ae7-8c18-4203dac08cb0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:52.967405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 765/1547] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:16:52.968624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e0c33-216b-4bdc-a3db-4d6e1ab32fe9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:52.969482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e0c33-216b-4bdc-a3db-4d6e1ab32fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:52.969756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e0c33-216b-4bdc-a3db-4d6e1ab32fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:52.997271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d72e0c33-216b-4bdc-a3db-4d6e1ab32fe9 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:52.998495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d72e0c33-216b-4bdc-a3db-4d6e1ab32fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.007808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 766/1548] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.008727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e0c33-216b-4bdc-a3db-4d6e1ab32fe9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.019632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e0c33-216b-4bdc-a3db-4d6e1ab32fe9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.024793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.025847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.026152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.026405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.027078 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:16:53.027384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.028999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.041792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 767/1549] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:16:53.042798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.051696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 783/1550] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:52 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:53.053234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9ac48-6d38-4b28-98ee-1fb8c3fb45e5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.053607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9ac48-6d38-4b28-98ee-1fb8c3fb45e5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.053953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9ac48-6d38-4b28-98ee-1fb8c3fb45e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.056946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b91d1-ccee-46f2-8df9-04e900abef0f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.074017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e4f9ac48-6d38-4b28-98ee-1fb8c3fb45e5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.075279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e4f9ac48-6d38-4b28-98ee-1fb8c3fb45e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.115828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.116434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.116720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.116992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.117614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.118619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:16:53.118619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Aug 23 08:16:53.118619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: Aug 23 08:16:53.119894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.135056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 784/1551] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.136390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90643c65-1cfd-4633-a5d0-400cc63cda9c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.136678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90643c65-1cfd-4633-a5d0-400cc63cda9c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.136952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90643c65-1cfd-4633-a5d0-400cc63cda9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.155923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-90643c65-1cfd-4633-a5d0-400cc63cda9c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.157575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-90643c65-1cfd-4633-a5d0-400cc63cda9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.165447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 785/1552] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.167547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef5f5-c990-4dbd-9edf-f5b01a0c118c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.167828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef5f5-c990-4dbd-9edf-f5b01a0c118c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.168100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef5f5-c990-4dbd-9edf-f5b01a0c118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.186925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8ffef5f5-c990-4dbd-9edf-f5b01a0c118c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.188141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8ffef5f5-c990-4dbd-9edf-f5b01a0c118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.195992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 786/1553] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.197085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6252fc-a6cb-41a8-aad2-b0f2656ec594 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.197360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6252fc-a6cb-41a8-aad2-b0f2656ec594 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.197628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6252fc-a6cb-41a8-aad2-b0f2656ec594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.216777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4d6252fc-a6cb-41a8-aad2-b0f2656ec594 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.217957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4d6252fc-a6cb-41a8-aad2-b0f2656ec594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.226332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 787/1554] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.226898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 768/1555] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:53.227510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47d5c-053b-497f-88e7-66c3d39d5078 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.227649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.227827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47d5c-053b-497f-88e7-66c3d39d5078 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.228051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47d5c-053b-497f-88e7-66c3d39d5078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.251436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9be47d5c-053b-497f-88e7-66c3d39d5078 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.252620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9be47d5c-053b-497f-88e7-66c3d39d5078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.296220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc280220-017c-41a7-8ea4-04ddc45f423f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.300900 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a8503-d537-45dc-97e9-dd24b48f756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.301507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a8503-d537-45dc-97e9-dd24b48f756d admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.301787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a8503-d537-45dc-97e9-dd24b48f756d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.302059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a8503-d537-45dc-97e9-dd24b48f756d admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.302683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a8503-d537-45dc-97e9-dd24b48f756d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.304612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a8503-d537-45dc-97e9-dd24b48f756d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.312149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 788/1556] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.312966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be47d5c-053b-497f-88e7-66c3d39d5078 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.322409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be47d5c-053b-497f-88e7-66c3d39d5078 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.327242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.327855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.328116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.328890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.329545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.330627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:16:53.330627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 23 08:16:53.330627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: Aug 23 08:16:53.331885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.391542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 769/1557] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:53.396465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1ca6-97c3-4590-a753-c36aab743fbc None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.396465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1ca6-97c3-4590-a753-c36aab743fbc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.396465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae1ca6-97c3-4590-a753-c36aab743fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.401518 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:53.412291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2cae1ca6-97c3-4590-a753-c36aab743fbc None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.413494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2cae1ca6-97c3-4590-a753-c36aab743fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.420658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 789/1558] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:53.420658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.430695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51620fd9-9254-4528-a633-557b1ede87bd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.450804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:53.476740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 770/1559] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.477994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5019d-1c79-4e4a-9fe8-68f7886bb981 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.478293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5019d-1c79-4e4a-9fe8-68f7886bb981 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.478549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5019d-1c79-4e4a-9fe8-68f7886bb981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.488129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.488735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.489012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.489281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.489918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:16:53.490217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.491254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:16:53.491254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 23 08:16:53.491254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: Aug 23 08:16:53.492299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.498531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-64c5019d-1c79-4e4a-9fe8-68f7886bb981 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.499775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-64c5019d-1c79-4e4a-9fe8-68f7886bb981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.505462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 790/1560] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:16:53.506266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.514930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 771/1561] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.516421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee408a-c213-485d-a7b0-ce1a6e624ba4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.516989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee408a-c213-485d-a7b0-ce1a6e624ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.517507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee408a-c213-485d-a7b0-ce1a6e624ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.527024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7221d4-95b8-49f1-8659-c381a07770f9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.531619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da732fcb-6f05-4f3b-a1a4-f264f25f55cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.532215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da732fcb-6f05-4f3b-a1a4-f264f25f55cb admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.532490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da732fcb-6f05-4f3b-a1a4-f264f25f55cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.532847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da732fcb-6f05-4f3b-a1a4-f264f25f55cb admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.533591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da732fcb-6f05-4f3b-a1a4-f264f25f55cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.535552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da732fcb-6f05-4f3b-a1a4-f264f25f55cb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.537984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b6ee408a-c213-485d-a7b0-ce1a6e624ba4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.539287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b6ee408a-c213-485d-a7b0-ce1a6e624ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.547916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 772/1562] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.559906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e4aa8-1dbe-47f3-85c9-d8f53b12b249 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.559906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e4aa8-1dbe-47f3-85c9-d8f53b12b249 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.559906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e4aa8-1dbe-47f3-85c9-d8f53b12b249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.568173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-804e4aa8-1dbe-47f3-85c9-d8f53b12b249 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:53.569373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-804e4aa8-1dbe-47f3-85c9-d8f53b12b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:53.577449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 773/1563] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:53.578672 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e4aa8-1dbe-47f3-85c9-d8f53b12b249 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.588842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e4aa8-1dbe-47f3-85c9-d8f53b12b249 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.594576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c918c-ed24-4ff5-90df-6f055aeb2b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.594747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c918c-ed24-4ff5-90df-6f055aeb2b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.595061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c918c-ed24-4ff5-90df-6f055aeb2b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.595338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c918c-ed24-4ff5-90df-6f055aeb2b11 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.595984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c918c-ed24-4ff5-90df-6f055aeb2b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.597947 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c918c-ed24-4ff5-90df-6f055aeb2b11 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.645247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 791/1564] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:53.646095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da732fcb-6f05-4f3b-a1a4-f264f25f55cb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.655847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da732fcb-6f05-4f3b-a1a4-f264f25f55cb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.660319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf797f-4b03-45ae-983c-64ee3f023ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.660999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf797f-4b03-45ae-983c-64ee3f023ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.661281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf797f-4b03-45ae-983c-64ee3f023ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.661552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf797f-4b03-45ae-983c-64ee3f023ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.662168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf797f-4b03-45ae-983c-64ee3f023ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.663819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf797f-4b03-45ae-983c-64ee3f023ed4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.674330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 774/1565] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:53.678498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 792/1566] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:53.678611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c918c-ed24-4ff5-90df-6f055aeb2b11 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.678726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf797f-4b03-45ae-983c-64ee3f023ed4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.685271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c918c-ed24-4ff5-90df-6f055aeb2b11 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.686415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf797f-4b03-45ae-983c-64ee3f023ed4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.691923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.692582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.692981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.693302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.693944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:16:53.694245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.695053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d5e9-6cbd-4822-812c-a9753a4294d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.695655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d5e9-6cbd-4822-812c-a9753a4294d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.695937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d5e9-6cbd-4822-812c-a9753a4294d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.696122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.696216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d5e9-6cbd-4822-812c-a9753a4294d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.696818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d5e9-6cbd-4822-812c-a9753a4294d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.698755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d5e9-6cbd-4822-812c-a9753a4294d6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.710117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 775/1567] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:16:53.711317 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.720829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a81d42-4e52-4bd9-b81f-d59bd9df0493 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.725629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.729798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.729798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.729798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.729798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:16:53.729798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.729798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.740815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 776/1568] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:16:53.741626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.751311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143eb21a-4311-4827-b980-845b5e2f6a03 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.756154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.758501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.758501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.758501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.758501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:16:53.758501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.759938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.771992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 777/1569] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:16:53.772801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.782722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d87e2-da71-4882-9a60-c99e5e80d7ca admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.787444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.788038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.793098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.793098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.793098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:16:53.793098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.793534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 793/1570] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:53.793534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d5e9-6cbd-4822-812c-a9753a4294d6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.793686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.805886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 778/1571] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:16:53.807036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.818098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f12775-e19c-4161-ac48-76adb9fca7b6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.854768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d5e9-6cbd-4822-812c-a9753a4294d6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.859388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba23636-c20f-424b-a83b-0b691953e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.860033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba23636-c20f-424b-a83b-0b691953e0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.860312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba23636-c20f-424b-a83b-0b691953e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.860584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba23636-c20f-424b-a83b-0b691953e0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.861225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba23636-c20f-424b-a83b-0b691953e0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.862832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba23636-c20f-424b-a83b-0b691953e0aa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.872429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 794/1572] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:53.873209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba23636-c20f-424b-a83b-0b691953e0aa admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.874840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eecb5-46ad-46d2-ba4c-af60ae018878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.876204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eecb5-46ad-46d2-ba4c-af60ae018878 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.876487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eecb5-46ad-46d2-ba4c-af60ae018878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.876760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eecb5-46ad-46d2-ba4c-af60ae018878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.877404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14eecb5-46ad-46d2-ba4c-af60ae018878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.879034 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14eecb5-46ad-46d2-ba4c-af60ae018878 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.882949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba23636-c20f-424b-a83b-0b691953e0aa admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.887488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358369e-3927-44a7-88a6-821506154f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.888047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358369e-3927-44a7-88a6-821506154f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.888325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358369e-3927-44a7-88a6-821506154f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.888597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358369e-3927-44a7-88a6-821506154f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.888998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 779/1573] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:53.889213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358369e-3927-44a7-88a6-821506154f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.889776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eecb5-46ad-46d2-ba4c-af60ae018878 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.890840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358369e-3927-44a7-88a6-821506154f35 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.899866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eecb5-46ad-46d2-ba4c-af60ae018878 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.901529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 795/1574] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:53.902237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358369e-3927-44a7-88a6-821506154f35 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:53.907502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eae5e-b850-479e-8910-dc57c80513fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.908094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eae5e-b850-479e-8910-dc57c80513fd admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.908376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eae5e-b850-479e-8910-dc57c80513fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.908650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eae5e-b850-479e-8910-dc57c80513fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.909297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5eae5e-b850-479e-8910-dc57c80513fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.911231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5eae5e-b850-479e-8910-dc57c80513fd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:53.913657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358369e-3927-44a7-88a6-821506154f35 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:53.918585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6c5c8-801d-4d0b-a75e-ef432d8ba03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:53.919150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6c5c8-801d-4d0b-a75e-ef432d8ba03f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:53.919430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6c5c8-801d-4d0b-a75e-ef432d8ba03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:53.919699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6c5c8-801d-4d0b-a75e-ef432d8ba03f admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:53.920309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6c5c8-801d-4d0b-a75e-ef432d8ba03f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:53.921444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:53.922179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6c5c8-801d-4d0b-a75e-ef432d8ba03f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.033509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 780/1575] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.034447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eae5e-b850-479e-8910-dc57c80513fd admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.049180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 796/1576] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:53 2022] POST /identity/v3/projects => generated 393 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.049180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6c5c8-801d-4d0b-a75e-ef432d8ba03f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.080993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:54.099163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eae5e-b850-479e-8910-dc57c80513fd admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.111710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6c5c8-801d-4d0b-a75e-ef432d8ba03f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.116388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3edff2-201a-464a-b3ed-ba3d53325098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.116991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3edff2-201a-464a-b3ed-ba3d53325098 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.117271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3edff2-201a-464a-b3ed-ba3d53325098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.117543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3edff2-201a-464a-b3ed-ba3d53325098 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.118168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3edff2-201a-464a-b3ed-ba3d53325098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.120201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3edff2-201a-464a-b3ed-ba3d53325098 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.155992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f67a4f-99a1-4f5e-8237-6195b294ca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.156647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f67a4f-99a1-4f5e-8237-6195b294ca00 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.156924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f67a4f-99a1-4f5e-8237-6195b294ca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.157195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f67a4f-99a1-4f5e-8237-6195b294ca00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.157808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f67a4f-99a1-4f5e-8237-6195b294ca00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.159754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f67a4f-99a1-4f5e-8237-6195b294ca00 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.250048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 797/1577] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:54 2022] POST /identity/v3/users => generated 295 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.251602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3edff2-201a-464a-b3ed-ba3d53325098 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.261310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3edff2-201a-464a-b3ed-ba3d53325098 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.266299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81b5a8-d1fc-4fb5-9899-3d808a65a958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.267063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81b5a8-d1fc-4fb5-9899-3d808a65a958 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.279298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81b5a8-d1fc-4fb5-9899-3d808a65a958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.279298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81b5a8-d1fc-4fb5-9899-3d808a65a958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.279298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81b5a8-d1fc-4fb5-9899-3d808a65a958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.279298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81b5a8-d1fc-4fb5-9899-3d808a65a958 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.279716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 781/1578] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:54 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.279716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f67a4f-99a1-4f5e-8237-6195b294ca00 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.335883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f67a4f-99a1-4f5e-8237-6195b294ca00 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.340829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e3b23-a5a6-4441-9921-e5458954fb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.341460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3b23-a5a6-4441-9921-e5458954fb82 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.341746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3b23-a5a6-4441-9921-e5458954fb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.342016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3b23-a5a6-4441-9921-e5458954fb82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.342658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e3b23-a5a6-4441-9921-e5458954fb82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.344853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e3b23-a5a6-4441-9921-e5458954fb82 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.346981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 798/1579] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:54 2022] POST /identity/v3/projects => generated 357 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.347770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81b5a8-d1fc-4fb5-9899-3d808a65a958 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.359182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 782/1580] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:54.359182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e3b23-a5a6-4441-9921-e5458954fb82 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.369434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e3b23-a5a6-4441-9921-e5458954fb82 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.382963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:54.409703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81b5a8-d1fc-4fb5-9899-3d808a65a958 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.414396 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625d6a8-8e52-4d56-8f01-27ab5e0799db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.415043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625d6a8-8e52-4d56-8f01-27ab5e0799db admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.415321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625d6a8-8e52-4d56-8f01-27ab5e0799db admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.415624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625d6a8-8e52-4d56-8f01-27ab5e0799db admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.416342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625d6a8-8e52-4d56-8f01-27ab5e0799db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.417836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625d6a8-8e52-4d56-8f01-27ab5e0799db admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.428076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 799/1581] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:54.428857 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625d6a8-8e52-4d56-8f01-27ab5e0799db admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.431837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.432509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.432792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.433068 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] PATH_INFO: `/v3/projects/c49d93c1beec4a329469f8f8d75d9af0/users/5305d30dae344365b84d9d79dbb24338/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.438176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625d6a8-8e52-4d56-8f01-27ab5e0799db admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.442764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb1053-058f-4b34-a0ca-8097cf154f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.443374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1053-058f-4b34-a0ca-8097cf154f6e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.443658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1053-058f-4b34-a0ca-8097cf154f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.443932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb1053-058f-4b34-a0ca-8097cf154f6e admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.444541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb1053-058f-4b34-a0ca-8097cf154f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.446735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb1053-058f-4b34-a0ca-8097cf154f6e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.448681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49d93c1beec4a329469f8f8d75d9af0, user_id=5305d30dae344365b84d9d79dbb24338, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.449703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:16:54.449703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Aug 23 08:16:54.449703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: Aug 23 08:16:54.451695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.517889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 783/1582] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:54 2022] PUT /identity/v3/projects/c49d93c1beec4a329469f8f8d75d9af0/users/5305d30dae344365b84d9d79dbb24338/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:54.518624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.556727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 800/1583] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:54 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.556727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb1053-058f-4b34-a0ca-8097cf154f6e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.564529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb1053-058f-4b34-a0ca-8097cf154f6e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.585295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f42587-f865-4432-bc1f-2d972e733ee6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.590072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c7856-30ae-4a7f-9c2a-f374fdd101da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.590673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7856-30ae-4a7f-9c2a-f374fdd101da admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.590973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7856-30ae-4a7f-9c2a-f374fdd101da admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.591249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c7856-30ae-4a7f-9c2a-f374fdd101da admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.591862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c7856-30ae-4a7f-9c2a-f374fdd101da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.593774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c7856-30ae-4a7f-9c2a-f374fdd101da admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.621581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc90864-f49e-4d92-b34a-f352c0cbf5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.622320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc90864-f49e-4d92-b34a-f352c0cbf5fe admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.622464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc90864-f49e-4d92-b34a-f352c0cbf5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.622736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc90864-f49e-4d92-b34a-f352c0cbf5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.623431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc90864-f49e-4d92-b34a-f352c0cbf5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.625097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc90864-f49e-4d92-b34a-f352c0cbf5fe admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.634769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 801/1584] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:54.635545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc90864-f49e-4d92-b34a-f352c0cbf5fe admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.644875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc90864-f49e-4d92-b34a-f352c0cbf5fe admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.649517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d10c72-3d90-4ffe-ba87-90758deb71f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.651018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d10c72-3d90-4ffe-ba87-90758deb71f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.651018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d10c72-3d90-4ffe-ba87-90758deb71f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.651018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d10c72-3d90-4ffe-ba87-90758deb71f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.651301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d10c72-3d90-4ffe-ba87-90758deb71f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.653053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d10c72-3d90-4ffe-ba87-90758deb71f7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.663125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:54.673682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 784/1585] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:54 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.676954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c7856-30ae-4a7f-9c2a-f374fdd101da admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.683936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c7856-30ae-4a7f-9c2a-f374fdd101da admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.688620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01e194-dbdf-4666-a68f-885e4eb46da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.689168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01e194-dbdf-4666-a68f-885e4eb46da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.689446 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01e194-dbdf-4666-a68f-885e4eb46da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.689715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01e194-dbdf-4666-a68f-885e4eb46da0 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.690329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01e194-dbdf-4666-a68f-885e4eb46da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.692248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01e194-dbdf-4666-a68f-885e4eb46da0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.774507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 785/1586] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:54 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.774963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 802/1587] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:54 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:54.775613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01e194-dbdf-4666-a68f-885e4eb46da0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.776646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d10c72-3d90-4ffe-ba87-90758deb71f7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.787717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01e194-dbdf-4666-a68f-885e4eb46da0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.791174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d10c72-3d90-4ffe-ba87-90758deb71f7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:54.792756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4fdf6-316d-47bb-bed9-6ec7affeb8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.793439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4fdf6-316d-47bb-bed9-6ec7affeb8be admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.793719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4fdf6-316d-47bb-bed9-6ec7affeb8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.793991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4fdf6-316d-47bb-bed9-6ec7affeb8be admin admin] PATH_INFO: `/v3/projects/7d310b6c64154e3bb5d95f1e762e3e80/users/dd81f4241d7343e990aea15e49be5f81/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.796316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3fcb6-0180-4f91-a649-ef9448d6ec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:54.796982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3fcb6-0180-4f91-a649-ef9448d6ec15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.797268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3fcb6-0180-4f91-a649-ef9448d6ec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.797540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3fcb6-0180-4f91-a649-ef9448d6ec15 admin admin] PATH_INFO: `/v3/projects/55ff9b7ff3b5483891849c10c99e2762/users/d19b147e3690496992ca2a7233ce7768/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.809052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4fdf6-316d-47bb-bed9-6ec7affeb8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d310b6c64154e3bb5d95f1e762e3e80, user_id=dd81f4241d7343e990aea15e49be5f81, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.811768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4fdf6-316d-47bb-bed9-6ec7affeb8be admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.812461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3fcb6-0180-4f91-a649-ef9448d6ec15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ff9b7ff3b5483891849c10c99e2762, user_id=d19b147e3690496992ca2a7233ce7768, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:54.815228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3fcb6-0180-4f91-a649-ef9448d6ec15 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:54.907681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 786/1588] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:54 2022] PUT /identity/v3/projects/7d310b6c64154e3bb5d95f1e762e3e80/users/dd81f4241d7343e990aea15e49be5f81/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:54.908914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c158d-c78b-4743-94d3-9a03cae19732 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:54.909200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c158d-c78b-4743-94d3-9a03cae19732 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:54.909471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c158d-c78b-4743-94d3-9a03cae19732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:54.935769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-516c158d-c78b-4743-94d3-9a03cae19732 None None] MFA Rules not processed for user `5305d30dae344365b84d9d79dbb24338`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:54.936980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-516c158d-c78b-4743-94d3-9a03cae19732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:54.941866 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 803/1589] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:54 2022] PUT /identity/v3/projects/55ff9b7ff3b5483891849c10c99e2762/users/d19b147e3690496992ca2a7233ce7768/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:54.955396 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3fcb6-0180-4f91-a649-ef9448d6ec15 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:54.979937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:55.004782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3fcb6-0180-4f91-a649-ef9448d6ec15 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.006454 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 787/1590] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:55.007410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c158d-c78b-4743-94d3-9a03cae19732 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.010048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5c5d1-2dbd-437f-a6f7-84ef149947c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.010726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5c5d1-2dbd-437f-a6f7-84ef149947c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.011049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5c5d1-2dbd-437f-a6f7-84ef149947c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.017879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5c5d1-2dbd-437f-a6f7-84ef149947c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.018573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5c5d1-2dbd-437f-a6f7-84ef149947c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.018719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c158d-c78b-4743-94d3-9a03cae19732 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.020216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5c5d1-2dbd-437f-a6f7-84ef149947c6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.023418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c43d2-0300-47ee-be23-7100af3d9601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.025862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c43d2-0300-47ee-be23-7100af3d9601 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.025862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c43d2-0300-47ee-be23-7100af3d9601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.025862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c43d2-0300-47ee-be23-7100af3d9601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.025862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c43d2-0300-47ee-be23-7100af3d9601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.027090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c43d2-0300-47ee-be23-7100af3d9601 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.035356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 804/1591] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.036258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5c5d1-2dbd-437f-a6f7-84ef149947c6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.041089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 788/1592] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.041925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c43d2-0300-47ee-be23-7100af3d9601 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.046781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5c5d1-2dbd-437f-a6f7-84ef149947c6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.052722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb235-07cb-4553-8e90-26a26df54def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.053319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb235-07cb-4553-8e90-26a26df54def admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.054475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c43d2-0300-47ee-be23-7100af3d9601 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.054587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb235-07cb-4553-8e90-26a26df54def admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.054587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb235-07cb-4553-8e90-26a26df54def admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.054587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cb235-07cb-4553-8e90-26a26df54def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.056113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cb235-07cb-4553-8e90-26a26df54def admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.058287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306176bf-bd0c-47cc-9474-c1ccd3999035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.058875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306176bf-bd0c-47cc-9474-c1ccd3999035 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.059254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306176bf-bd0c-47cc-9474-c1ccd3999035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.059525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306176bf-bd0c-47cc-9474-c1ccd3999035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.060202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306176bf-bd0c-47cc-9474-c1ccd3999035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.061862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306176bf-bd0c-47cc-9474-c1ccd3999035 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.066772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 805/1593] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.067629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb235-07cb-4553-8e90-26a26df54def admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.072441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 789/1594] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.073651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d45f1-cf06-4f5d-b114-c855a4803aa0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.073939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d45f1-cf06-4f5d-b114-c855a4803aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.074209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d45f1-cf06-4f5d-b114-c855a4803aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.079917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb235-07cb-4553-8e90-26a26df54def admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.085481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9045b-3bb6-4070-869e-e85391742ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.093275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9045b-3bb6-4070-869e-e85391742ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.093275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9045b-3bb6-4070-869e-e85391742ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.093275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9045b-3bb6-4070-869e-e85391742ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.093275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9045b-3bb6-4070-869e-e85391742ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.093275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9045b-3bb6-4070-869e-e85391742ed0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.093829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ad8d45f1-cf06-4f5d-b114-c855a4803aa0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:55.095238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ad8d45f1-cf06-4f5d-b114-c855a4803aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:55.101136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 806/1595] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.102356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448760d7-7480-4e3d-8398-f858b2e9d5f3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.102643 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448760d7-7480-4e3d-8398-f858b2e9d5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.102920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448760d7-7480-4e3d-8398-f858b2e9d5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.105979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 790/1596] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:55.106789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d45f1-cf06-4f5d-b114-c855a4803aa0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.118688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d45f1-cf06-4f5d-b114-c855a4803aa0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.124033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491080-5ad1-4f80-b518-b31eec79340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.124632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491080-5ad1-4f80-b518-b31eec79340d admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.124908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491080-5ad1-4f80-b518-b31eec79340d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.125712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491080-5ad1-4f80-b518-b31eec79340d admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.126366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491080-5ad1-4f80-b518-b31eec79340d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.127996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491080-5ad1-4f80-b518-b31eec79340d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.136330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-448760d7-7480-4e3d-8398-f858b2e9d5f3 None None] MFA Rules not processed for user `d19b147e3690496992ca2a7233ce7768`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:55.137534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-448760d7-7480-4e3d-8398-f858b2e9d5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:55.138177 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 791/1597] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.139398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6f0e-2e9c-47fb-aeb4-e9e5cf8dcd80 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.139675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6f0e-2e9c-47fb-aeb4-e9e5cf8dcd80 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.139945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6f0e-2e9c-47fb-aeb4-e9e5cf8dcd80 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.142024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 792/1598] 10.222.0.58 () {54 vars in 962 bytes} [Tue Aug 23 08:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:16:55.142502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e6f0e-2e9c-47fb-aeb4-e9e5cf8dcd80 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.155723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e6f0e-2e9c-47fb-aeb4-e9e5cf8dcd80 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.160226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cdc6c-846e-4d36-b48d-1de903915ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.160881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cdc6c-846e-4d36-b48d-1de903915ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.161151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cdc6c-846e-4d36-b48d-1de903915ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.161425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cdc6c-846e-4d36-b48d-1de903915ea7 admin admin] PATH_INFO: `/v3/projects/bc3167dc177744959760f39d57937670/users/ce316aa5c0b54ac99ade6cc4163ee209/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.175967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cdc6c-846e-4d36-b48d-1de903915ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3167dc177744959760f39d57937670, user_id=ce316aa5c0b54ac99ade6cc4163ee209, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.178681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cdc6c-846e-4d36-b48d-1de903915ea7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.205428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 807/1599] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:55.206305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448760d7-7480-4e3d-8398-f858b2e9d5f3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.216286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448760d7-7480-4e3d-8398-f858b2e9d5f3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.225452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bb2f4-97ea-4538-8c07-953481b2c4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.225452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bb2f4-97ea-4538-8c07-953481b2c4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.225452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bb2f4-97ea-4538-8c07-953481b2c4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.225452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bb2f4-97ea-4538-8c07-953481b2c4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.225866 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bb2f4-97ea-4538-8c07-953481b2c4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.225866 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bb2f4-97ea-4538-8c07-953481b2c4f9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.233632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 808/1600] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.234388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bb2f4-97ea-4538-8c07-953481b2c4f9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.243512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bb2f4-97ea-4538-8c07-953481b2c4f9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.248256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.248815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.249089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.249358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.249967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.252108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 793/1601] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:55 2022] PUT /identity/v3/projects/bc3167dc177744959760f39d57937670/users/ce316aa5c0b54ac99ade6cc4163ee209/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:55.252108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cdc6c-846e-4d36-b48d-1de903915ea7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.252338 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.262489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 809/1602] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.263320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.264983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:55.278659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:55.315413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cdc6c-846e-4d36-b48d-1de903915ea7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.320520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec95c1-ed25-4d47-8141-89dcb9281ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.321122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec95c1-ed25-4d47-8141-89dcb9281ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.321401 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec95c1-ed25-4d47-8141-89dcb9281ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.321668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec95c1-ed25-4d47-8141-89dcb9281ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.321822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1f47-b56c-474f-a697-80fde0cfbbed admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.322307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec95c1-ed25-4d47-8141-89dcb9281ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.323937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec95c1-ed25-4d47-8141-89dcb9281ae6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.334382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 794/1603] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:55.335028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec95c1-ed25-4d47-8141-89dcb9281ae6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.345178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec95c1-ed25-4d47-8141-89dcb9281ae6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.350333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf96320-c68b-4b38-9c42-a24d476558b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.351118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf96320-c68b-4b38-9c42-a24d476558b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.351523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf96320-c68b-4b38-9c42-a24d476558b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.351891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf96320-c68b-4b38-9c42-a24d476558b5 admin admin] PATH_INFO: `/v3/projects/df0b6e7c14ba4a4eba256ad043f52c5d/users/e1ad75433eca4c3e8ec6cbc6fa5cc79e/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.366919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf96320-c68b-4b38-9c42-a24d476558b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0b6e7c14ba4a4eba256ad043f52c5d, user_id=e1ad75433eca4c3e8ec6cbc6fa5cc79e, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.369699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf96320-c68b-4b38-9c42-a24d476558b5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.373163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54cdd3-7a67-478b-81c4-6c53981b8ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.373769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54cdd3-7a67-478b-81c4-6c53981b8ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.374114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54cdd3-7a67-478b-81c4-6c53981b8ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.374389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54cdd3-7a67-478b-81c4-6c53981b8ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.383592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad54cdd3-7a67-478b-81c4-6c53981b8ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.385954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad54cdd3-7a67-478b-81c4-6c53981b8ad1 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.394515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 810/1604] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:55.395713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d5d1-1b85-4340-8e7d-b62b46f9e903 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.396066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d5d1-1b85-4340-8e7d-b62b46f9e903 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.396401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95d5d1-1b85-4340-8e7d-b62b46f9e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.433648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 795/1605] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:55 2022] PUT /identity/v3/projects/df0b6e7c14ba4a4eba256ad043f52c5d/users/e1ad75433eca4c3e8ec6cbc6fa5cc79e/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:55.434474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf96320-c68b-4b38-9c42-a24d476558b5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.436880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0e95d5d1-1b85-4340-8e7d-b62b46f9e903 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:55.438218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e95d5d1-1b85-4340-8e7d-b62b46f9e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:55.491840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:55.505579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf96320-c68b-4b38-9c42-a24d476558b5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.507371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 811/1606] 10.222.0.58 () {58 vars in 1059 bytes} [Tue Aug 23 08:16:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:55.508199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95d5d1-1b85-4340-8e7d-b62b46f9e903 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.510482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb23a2-0a39-4fa4-8a4f-5221780deceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.511106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb23a2-0a39-4fa4-8a4f-5221780deceb admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.511475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb23a2-0a39-4fa4-8a4f-5221780deceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.511717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb23a2-0a39-4fa4-8a4f-5221780deceb admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.512321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb23a2-0a39-4fa4-8a4f-5221780deceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.514327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb23a2-0a39-4fa4-8a4f-5221780deceb admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.519528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95d5d1-1b85-4340-8e7d-b62b46f9e903 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.524529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cbf11-8240-4eab-869e-a90d82a8d715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.525308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cbf11-8240-4eab-869e-a90d82a8d715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.525493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cbf11-8240-4eab-869e-a90d82a8d715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.525767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cbf11-8240-4eab-869e-a90d82a8d715 admin admin] PATH_INFO: `/v3/projects/cfd891d8d1b7436fabfa2c48019c3565/users/b3c984088c0448f8abc0a0b60b820b3e/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.542597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cbf11-8240-4eab-869e-a90d82a8d715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd891d8d1b7436fabfa2c48019c3565, user_id=b3c984088c0448f8abc0a0b60b820b3e, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.545352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cbf11-8240-4eab-869e-a90d82a8d715 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.639933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 796/1607] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:55 2022] POST /identity/v3/projects => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:55.641850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26a10-c6c5-40e6-b6e5-73af78967bee None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.642779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26a10-c6c5-40e6-b6e5-73af78967bee None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.642779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26a10-c6c5-40e6-b6e5-73af78967bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.661278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-91e26a10-c6c5-40e6-b6e5-73af78967bee None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:55.662927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-91e26a10-c6c5-40e6-b6e5-73af78967bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:55.667503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 812/1608] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:55 2022] PUT /identity/v3/projects/cfd891d8d1b7436fabfa2c48019c3565/users/b3c984088c0448f8abc0a0b60b820b3e/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:55.668778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cbf11-8240-4eab-869e-a90d82a8d715 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.694658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:55.728454 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 797/1609] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:55.733261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26a10-c6c5-40e6-b6e5-73af78967bee None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.734863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cbf11-8240-4eab-869e-a90d82a8d715 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.740452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30703d93-ef59-4a80-8cc5-446b7e2cbaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.740649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30703d93-ef59-4a80-8cc5-446b7e2cbaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.740945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30703d93-ef59-4a80-8cc5-446b7e2cbaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.741223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30703d93-ef59-4a80-8cc5-446b7e2cbaa4 admin admin] PATH_INFO: `/v3/projects/231e7186d1fd4a89aac442db1e8db47f/users/6bf59213ab0648c996a1da73964afea8/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.741363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26a10-c6c5-40e6-b6e5-73af78967bee None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:55.746883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be16025-b0cc-4b41-b1e6-e046335034f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:55.747721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be16025-b0cc-4b41-b1e6-e046335034f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.748463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be16025-b0cc-4b41-b1e6-e046335034f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.748463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be16025-b0cc-4b41-b1e6-e046335034f7 admin admin] PATH_INFO: `/v3/projects/6e70945a174d4af4abedd30925eb4130/users/e53e9053a81f4e8884df1e50cd778305/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.759161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30703d93-ef59-4a80-8cc5-446b7e2cbaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231e7186d1fd4a89aac442db1e8db47f, user_id=6bf59213ab0648c996a1da73964afea8, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.763295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30703d93-ef59-4a80-8cc5-446b7e2cbaa4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.767672 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be16025-b0cc-4b41-b1e6-e046335034f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e70945a174d4af4abedd30925eb4130, user_id=e53e9053a81f4e8884df1e50cd778305, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:55.770394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be16025-b0cc-4b41-b1e6-e046335034f7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:55.949854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 798/1611] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:55 2022] PUT /identity/v3/projects/6e70945a174d4af4abedd30925eb4130/users/e53e9053a81f4e8884df1e50cd778305/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:55.950075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 813/1611] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:55 2022] PUT /identity/v3/projects/231e7186d1fd4a89aac442db1e8db47f/users/6bf59213ab0648c996a1da73964afea8/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:55.950075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30703d93-ef59-4a80-8cc5-446b7e2cbaa4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:55.950683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6cac0-c1df-48b6-a4c6-cb711cbeebc8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:55.950683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6cac0-c1df-48b6-a4c6-cb711cbeebc8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:55.950683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6cac0-c1df-48b6-a4c6-cb711cbeebc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:55.979915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0ec6cac0-c1df-48b6-a4c6-cb711cbeebc8 None None] MFA Rules not processed for user `e1ad75433eca4c3e8ec6cbc6fa5cc79e`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:55.981073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0ec6cac0-c1df-48b6-a4c6-cb711cbeebc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:56.033595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30703d93-ef59-4a80-8cc5-446b7e2cbaa4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.041399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8245641-d69d-4c25-b973-cd41fa5e448e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.042268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245641-d69d-4c25-b973-cd41fa5e448e service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.042722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245641-d69d-4c25-b973-cd41fa5e448e service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.043178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245641-d69d-4c25-b973-cd41fa5e448e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.055097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 799/1612] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:56.055097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6cac0-c1df-48b6-a4c6-cb711cbeebc8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.107108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:56.140372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8245641-d69d-4c25-b973-cd41fa5e448e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.145135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6cac0-c1df-48b6-a4c6-cb711cbeebc8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.145411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8245641-d69d-4c25-b973-cd41fa5e448e service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.146772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b838d-7366-48a7-aee9-2803d2817c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.148200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b838d-7366-48a7-aee9-2803d2817c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.148581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b838d-7366-48a7-aee9-2803d2817c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.149370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b838d-7366-48a7-aee9-2803d2817c06 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.150222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b838d-7366-48a7-aee9-2803d2817c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.152596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 814/1613] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:56.152992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b838d-7366-48a7-aee9-2803d2817c06 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.153702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad246510-5b73-454d-88f6-25e1a9089a58 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.153992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad246510-5b73-454d-88f6-25e1a9089a58 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.154437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad246510-5b73-454d-88f6-25e1a9089a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.174476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ad246510-5b73-454d-88f6-25e1a9089a58 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:56.175758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ad246510-5b73-454d-88f6-25e1a9089a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:56.184163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 815/1614] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:56.184991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad246510-5b73-454d-88f6-25e1a9089a58 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.227767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:56.241627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad246510-5b73-454d-88f6-25e1a9089a58 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.245680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0ea62-a149-419c-b0f8-1814509f4638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.246292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ea62-a149-419c-b0f8-1814509f4638 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.246570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ea62-a149-419c-b0f8-1814509f4638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.246835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ea62-a149-419c-b0f8-1814509f4638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.256349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 800/1615] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:56 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:56.256349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc84b3-4e37-4e99-8ef6-317ea33ffe71 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.256349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc84b3-4e37-4e99-8ef6-317ea33ffe71 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.256349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc84b3-4e37-4e99-8ef6-317ea33ffe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.259349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0ea62-a149-419c-b0f8-1814509f4638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.261061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0ea62-a149-419c-b0f8-1814509f4638 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.277754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 816/1616] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:56.279410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0115b6-874d-4493-97f9-cc2ac08491f3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.279410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0115b6-874d-4493-97f9-cc2ac08491f3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.279410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0115b6-874d-4493-97f9-cc2ac08491f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.300657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3ddc84b3-4e37-4e99-8ef6-317ea33ffe71 None None] MFA Rules not processed for user `6bf59213ab0648c996a1da73964afea8`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:56.300657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3ddc84b3-4e37-4e99-8ef6-317ea33ffe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:56.314266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-df0115b6-874d-4493-97f9-cc2ac08491f3 None None] MFA Rules not processed for user `e53e9053a81f4e8884df1e50cd778305`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:56.315916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-df0115b6-874d-4493-97f9-cc2ac08491f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:56.405314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 801/1617] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:56.406372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242aa8-15ce-45f1-9d9e-5df1ebac94b7 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.406806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242aa8-15ce-45f1-9d9e-5df1ebac94b7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.407109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242aa8-15ce-45f1-9d9e-5df1ebac94b7 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.408521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 802/1618] 10.222.0.58 () {54 vars in 962 bytes} [Tue Aug 23 08:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:16:56.410582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 817/1619] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:56.411455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0115b6-874d-4493-97f9-cc2ac08491f3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.421508 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242aa8-15ce-45f1-9d9e-5df1ebac94b7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.436814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242aa8-15ce-45f1-9d9e-5df1ebac94b7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.437333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0115b6-874d-4493-97f9-cc2ac08491f3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.441186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cbc8c-f02e-42d9-bcef-dc6c7f33acee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.444302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cbc8c-f02e-42d9-bcef-dc6c7f33acee service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.444302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cbc8c-f02e-42d9-bcef-dc6c7f33acee service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.444302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cbc8c-f02e-42d9-bcef-dc6c7f33acee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.444679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68613dd-0645-4dc4-9d75-567c78f52dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.444679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68613dd-0645-4dc4-9d75-567c78f52dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.444679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68613dd-0645-4dc4-9d75-567c78f52dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.444679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68613dd-0645-4dc4-9d75-567c78f52dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.445029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68613dd-0645-4dc4-9d75-567c78f52dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.447067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68613dd-0645-4dc4-9d75-567c78f52dc9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.454289 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cbc8c-f02e-42d9-bcef-dc6c7f33acee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.455875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cbc8c-f02e-42d9-bcef-dc6c7f33acee service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.458350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 818/1620] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:56.459421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68613dd-0645-4dc4-9d75-567c78f52dc9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.467253 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 803/1621] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:56.468592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fcfb9-526e-45c3-900f-fc5cd6bef82e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.468868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fcfb9-526e-45c3-900f-fc5cd6bef82e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.471261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fcfb9-526e-45c3-900f-fc5cd6bef82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.472615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68613dd-0645-4dc4-9d75-567c78f52dc9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.478004 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81506515-301e-4995-bdf9-6af14774bb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.478750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506515-301e-4995-bdf9-6af14774bb01 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.479081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506515-301e-4995-bdf9-6af14774bb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.479356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81506515-301e-4995-bdf9-6af14774bb01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.480144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81506515-301e-4995-bdf9-6af14774bb01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.482089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81506515-301e-4995-bdf9-6af14774bb01 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.492430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-183fcfb9-526e-45c3-900f-fc5cd6bef82e None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:56.494075 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-183fcfb9-526e-45c3-900f-fc5cd6bef82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:56.505051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 804/1622] 10.222.0.58 () {58 vars in 1059 bytes} [Tue Aug 23 08:16:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:56.507817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fcfb9-526e-45c3-900f-fc5cd6bef82e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.519542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fcfb9-526e-45c3-900f-fc5cd6bef82e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.527514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466ca1d-2fb2-48b1-b9ca-1c89d5798948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.528125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466ca1d-2fb2-48b1-b9ca-1c89d5798948 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.528425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466ca1d-2fb2-48b1-b9ca-1c89d5798948 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.528695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466ca1d-2fb2-48b1-b9ca-1c89d5798948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.537191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466ca1d-2fb2-48b1-b9ca-1c89d5798948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.539031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466ca1d-2fb2-48b1-b9ca-1c89d5798948 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.548178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 805/1623] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:56.548612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466ca1d-2fb2-48b1-b9ca-1c89d5798948 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.558886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466ca1d-2fb2-48b1-b9ca-1c89d5798948 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.564509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51410d-d0b6-4ce5-8797-c1932e9bcef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.565165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51410d-d0b6-4ce5-8797-c1932e9bcef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.565443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51410d-d0b6-4ce5-8797-c1932e9bcef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.568758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51410d-d0b6-4ce5-8797-c1932e9bcef9 admin admin] PATH_INFO: `/v3/projects/2655779c0d4d465e8b7b3452ce95aabd/users/023015ede0b64d3bada5cb15a60b5efe/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.583901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51410d-d0b6-4ce5-8797-c1932e9bcef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2655779c0d4d465e8b7b3452ce95aabd, user_id=023015ede0b64d3bada5cb15a60b5efe, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.586579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51410d-d0b6-4ce5-8797-c1932e9bcef9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.619123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 819/1624] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:16:56 2022] POST /identity/v3/projects => generated 355 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:56.620199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81506515-301e-4995-bdf9-6af14774bb01 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.680871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 806/1625] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:56 2022] PUT /identity/v3/projects/2655779c0d4d465e8b7b3452ce95aabd/users/023015ede0b64d3bada5cb15a60b5efe/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:56.682112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d37e8-db85-4557-b7a5-2ed84b65bbee None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.682112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d37e8-db85-4557-b7a5-2ed84b65bbee None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.682335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d37e8-db85-4557-b7a5-2ed84b65bbee None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.683958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 807/1626] 10.222.0.58 () {54 vars in 912 bytes} [Tue Aug 23 08:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:16:56.684671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d37e8-db85-4557-b7a5-2ed84b65bbee None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.731447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81506515-301e-4995-bdf9-6af14774bb01 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.776496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d37e8-db85-4557-b7a5-2ed84b65bbee None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.786363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c54c62-c080-445a-95c2-e23b7e192f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.787087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c54c62-c080-445a-95c2-e23b7e192f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.787561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c54c62-c080-445a-95c2-e23b7e192f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.788047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:56.788713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c54c62-c080-445a-95c2-e23b7e192f44 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.789551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c54c62-c080-445a-95c2-e23b7e192f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.792456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c54c62-c080-445a-95c2-e23b7e192f44 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.802159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:56.837462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4b7bb-ba0b-45c7-8026-2d8df4c55295 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.838219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4b7bb-ba0b-45c7-8026-2d8df4c55295 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.838348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4b7bb-ba0b-45c7-8026-2d8df4c55295 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.838618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4b7bb-ba0b-45c7-8026-2d8df4c55295 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.886755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 808/1627] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:16:56 2022] POST /identity/v3/users => generated 284 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:16:56.887581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c54c62-c080-445a-95c2-e23b7e192f44 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.898204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c54c62-c080-445a-95c2-e23b7e192f44 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.903270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c2494-25f3-4ddb-8d40-d4fb8c07c9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.903886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c2494-25f3-4ddb-8d40-d4fb8c07c9ed admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.904329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c2494-25f3-4ddb-8d40-d4fb8c07c9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.904597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c2494-25f3-4ddb-8d40-d4fb8c07c9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.905237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c2494-25f3-4ddb-8d40-d4fb8c07c9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.907194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c2494-25f3-4ddb-8d40-d4fb8c07c9ed admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.907793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4b7bb-ba0b-45c7-8026-2d8df4c55295 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.909392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4b7bb-ba0b-45c7-8026-2d8df4c55295 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.918843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 820/1628] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:56.920268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec812fef-7893-42ed-b821-44ab3de87f72 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.920562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec812fef-7893-42ed-b821-44ab3de87f72 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.920836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec812fef-7893-42ed-b821-44ab3de87f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.929689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 809/1629] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:56.930290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c2494-25f3-4ddb-8d40-d4fb8c07c9ed admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.946226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c2494-25f3-4ddb-8d40-d4fb8c07c9ed admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.948211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ec812fef-7893-42ed-b821-44ab3de87f72 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:56.949489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ec812fef-7893-42ed-b821-44ab3de87f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:56.951536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c3d18-f4eb-458e-9677-271c580bdcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.952449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c3d18-f4eb-458e-9677-271c580bdcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.953113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c3d18-f4eb-458e-9677-271c580bdcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.953599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c3d18-f4eb-458e-9677-271c580bdcd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.955005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c3d18-f4eb-458e-9677-271c580bdcd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.956711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c3d18-f4eb-458e-9677-271c580bdcd8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:56.967071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 821/1630] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:16:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:56.967965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec812fef-7893-42ed-b821-44ab3de87f72 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.975861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 810/1631] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:56.981809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c3d18-f4eb-458e-9677-271c580bdcd8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:56.985918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec812fef-7893-42ed-b821-44ab3de87f72 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.989842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885ce3-731a-42fb-a78b-7a7abf6947d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:56.990556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885ce3-731a-42fb-a78b-7a7abf6947d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:56.991494 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885ce3-731a-42fb-a78b-7a7abf6947d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:56.991774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885ce3-731a-42fb-a78b-7a7abf6947d1 admin admin] PATH_INFO: `/v3/projects/2655779c0d4d465e8b7b3452ce95aabd/users/023015ede0b64d3bada5cb15a60b5efe/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:56.994688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885ce3-731a-42fb-a78b-7a7abf6947d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2655779c0d4d465e8b7b3452ce95aabd, user_id=023015ede0b64d3bada5cb15a60b5efe, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:56.995226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c3d18-f4eb-458e-9677-271c580bdcd8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:56.998055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885ce3-731a-42fb-a78b-7a7abf6947d1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:57.000311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbbe8e-fe74-4e23-9684-d820935408e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:57.001563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbbe8e-fe74-4e23-9684-d820935408e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.001846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbbe8e-fe74-4e23-9684-d820935408e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.002116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbbe8e-fe74-4e23-9684-d820935408e9 admin admin] PATH_INFO: `/v3/projects/c997742a305149afa395083a4eddce31/users/ed51541ab3f04a7491b07201007f2a9f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.226585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 822/1632] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:56 2022] PUT /identity/v3/projects/2655779c0d4d465e8b7b3452ce95aabd/users/023015ede0b64d3bada5cb15a60b5efe/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:57.229413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbbe8e-fe74-4e23-9684-d820935408e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c997742a305149afa395083a4eddce31, user_id=ed51541ab3f04a7491b07201007f2a9f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:57.229413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbbe8e-fe74-4e23-9684-d820935408e9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:57.230163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885ce3-731a-42fb-a78b-7a7abf6947d1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:57.230163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885ce3-731a-42fb-a78b-7a7abf6947d1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:57.230163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43d0e1-3b38-451f-9494-e1303cb22a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:57.230163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43d0e1-3b38-451f-9494-e1303cb22a5e service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.230671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43d0e1-3b38-451f-9494-e1303cb22a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.230671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43d0e1-3b38-451f-9494-e1303cb22a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.231187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 811/1633] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:56 2022] PUT /identity/v3/projects/c997742a305149afa395083a4eddce31/users/ed51541ab3f04a7491b07201007f2a9f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:57.231187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b0b4c-5a60-4ba1-8964-af90f6ce2150 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.231187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b0b4c-5a60-4ba1-8964-af90f6ce2150 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.231187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b0b4c-5a60-4ba1-8964-af90f6ce2150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.254764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:57.257251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-970b0b4c-5a60-4ba1-8964-af90f6ce2150 None None] MFA Rules not processed for user `023015ede0b64d3bada5cb15a60b5efe`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:57.258532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43d0e1-3b38-451f-9494-e1303cb22a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:57.258999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-970b0b4c-5a60-4ba1-8964-af90f6ce2150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:57.260167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43d0e1-3b38-451f-9494-e1303cb22a5e service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:57.275515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 823/1634] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:57.276379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43d0e1-3b38-451f-9494-e1303cb22a5e service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:57.365711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 812/1635] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:57.371845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2716-86a5-4c32-b4b6-169876d2aefe None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.372505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2716-86a5-4c32-b4b6-169876d2aefe None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.372722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2716-86a5-4c32-b4b6-169876d2aefe None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.378998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 813/1636] 10.222.0.58 () {54 vars in 912 bytes} [Tue Aug 23 08:16:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:16:57.379915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2716-86a5-4c32-b4b6-169876d2aefe None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:57.406486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43d0e1-3b38-451f-9494-e1303cb22a5e service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:57.492723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2716-86a5-4c32-b4b6-169876d2aefe None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:57.501275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91ea3b-a2c1-456d-b267-cd457b8949d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:57.501940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ea3b-a2c1-456d-b267-cd457b8949d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.502214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ea3b-a2c1-456d-b267-cd457b8949d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.503136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91ea3b-a2c1-456d-b267-cd457b8949d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.523019 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91ea3b-a2c1-456d-b267-cd457b8949d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:57.525937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91ea3b-a2c1-456d-b267-cd457b8949d0 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:57.536721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969be403-09a7-45a3-8e93-56c1c30b2a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:57.538146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969be403-09a7-45a3-8e93-56c1c30b2a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.538433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969be403-09a7-45a3-8e93-56c1c30b2a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.539865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969be403-09a7-45a3-8e93-56c1c30b2a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.541684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969be403-09a7-45a3-8e93-56c1c30b2a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:57.543789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969be403-09a7-45a3-8e93-56c1c30b2a62 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:57.563087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 814/1637] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:57.571761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eccfe9-7be8-4ef2-a0c1-20ad9e31e888 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.572192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eccfe9-7be8-4ef2-a0c1-20ad9e31e888 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.572581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eccfe9-7be8-4ef2-a0c1-20ad9e31e888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.583134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 824/1638] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:16:57.585825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969be403-09a7-45a3-8e93-56c1c30b2a62 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:57.599113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-58eccfe9-7be8-4ef2-a0c1-20ad9e31e888 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:57.600732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-58eccfe9-7be8-4ef2-a0c1-20ad9e31e888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:57.609132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969be403-09a7-45a3-8e93-56c1c30b2a62 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:57.619077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a9990-efb7-4efd-bde6-bf35c166dc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:57.619748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9990-efb7-4efd-bde6-bf35c166dc85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.620467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9990-efb7-4efd-bde6-bf35c166dc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.622057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9990-efb7-4efd-bde6-bf35c166dc85 admin admin] PATH_INFO: `/v3/projects/c997742a305149afa395083a4eddce31/users/ed51541ab3f04a7491b07201007f2a9f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.626302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a9990-efb7-4efd-bde6-bf35c166dc85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c997742a305149afa395083a4eddce31, user_id=ed51541ab3f04a7491b07201007f2a9f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:57.629828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a9990-efb7-4efd-bde6-bf35c166dc85 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:57.639167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:57.653906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 825/1639] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:16:57 2022] PUT /identity/v3/projects/c997742a305149afa395083a4eddce31/users/ed51541ab3f04a7491b07201007f2a9f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:16:57.656032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e894b2-bb4f-4331-9b8d-8de35d518453 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.656422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e894b2-bb4f-4331-9b8d-8de35d518453 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.656907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e894b2-bb4f-4331-9b8d-8de35d518453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.691415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-85e894b2-bb4f-4331-9b8d-8de35d518453 None None] MFA Rules not processed for user `ed51541ab3f04a7491b07201007f2a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:16:57.692889 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-85e894b2-bb4f-4331-9b8d-8de35d518453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:16:57.714768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 815/1640] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:16:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:57.766979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eccfe9-7be8-4ef2-a0c1-20ad9e31e888 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:57.782407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eccfe9-7be8-4ef2-a0c1-20ad9e31e888 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:57.844875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 826/1641] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:16:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:16:57.862594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614128b-7ef7-4abd-a525-8101c1130bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:57.863220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614128b-7ef7-4abd-a525-8101c1130bcf service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:57.863493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614128b-7ef7-4abd-a525-8101c1130bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:57.863761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614128b-7ef7-4abd-a525-8101c1130bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:57.946991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614128b-7ef7-4abd-a525-8101c1130bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:57.948388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614128b-7ef7-4abd-a525-8101c1130bcf service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:57.971389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 816/1642] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:16:58.784992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e894b2-bb4f-4331-9b8d-8de35d518453 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:58.806487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e894b2-bb4f-4331-9b8d-8de35d518453 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:58.814741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae3c63-ffc2-4ca7-ba24-0a50189378ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:58.816856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae3c63-ffc2-4ca7-ba24-0a50189378ed service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:58.818820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae3c63-ffc2-4ca7-ba24-0a50189378ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:58.823707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae3c63-ffc2-4ca7-ba24-0a50189378ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:58.847885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:16:58.855181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae3c63-ffc2-4ca7-ba24-0a50189378ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:58.856667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae3c63-ffc2-4ca7-ba24-0a50189378ed service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:58.913660 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 827/1643] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:16:59.670611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614128b-7ef7-4abd-a525-8101c1130bcf service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:16:59.684273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614128b-7ef7-4abd-a525-8101c1130bcf service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:16:59.689808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9be1f-edff-4d75-b510-c4385dd14c78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:16:59.690430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9be1f-edff-4d75-b510-c4385dd14c78 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:16:59.690764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9be1f-edff-4d75-b510-c4385dd14c78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:16:59.691069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9be1f-edff-4d75-b510-c4385dd14c78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:16:59.883427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9be1f-edff-4d75-b510-c4385dd14c78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:16:59.884987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9be1f-edff-4d75-b510-c4385dd14c78 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:16:59.908767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 817/1644] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:16:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:00.198654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae3c63-ffc2-4ca7-ba24-0a50189378ed service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:00.320465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae3c63-ffc2-4ca7-ba24-0a50189378ed service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:00.326372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:00.327259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:00.327664 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:00.328089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] PATH_INFO: `/v3/users/5305d30dae344365b84d9d79dbb24338` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:00.329876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5305d30dae344365b84d9d79dbb24338)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:00.332304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:00.607680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5305d30dae344365b84d9d79dbb24338'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:00.629666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5305d30dae344365b84d9d79dbb24338'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:00.630777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5305d30dae344365b84d9d79dbb24338'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:00.633824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5305d30dae344365b84d9d79dbb24338'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:00.687112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5305d30dae344365b84d9d79dbb24338'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:01.003058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 828/1645] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:00 2022] DELETE /identity/v3/users/5305d30dae344365b84d9d79dbb24338 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:01.007830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7e714-7d36-47f7-a4c4-0a4db7688a3f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:01.008406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7e714-7d36-47f7-a4c4-0a4db7688a3f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:01.008406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7e714-7d36-47f7-a4c4-0a4db7688a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:01.018181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:01.060360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e8d7e714-7d36-47f7-a4c4-0a4db7688a3f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:01.065940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e8d7e714-7d36-47f7-a4c4-0a4db7688a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:01.222215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 818/1646] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:01.235952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:01.240280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7e714-7d36-47f7-a4c4-0a4db7688a3f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:01.242291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e8d7e714-7d36-47f7-a4c4-0a4db7688a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:01.243322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:01.269106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7e714-7d36-47f7-a4c4-0a4db7688a3f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:01.282850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:01.283954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:01.284361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:01.284778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] PATH_INFO: `/v3/users/d19b147e3690496992ca2a7233ce7768` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:01.286096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d19b147e3690496992ca2a7233ce7768)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:01.287314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: WARNING py.warnings [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:17:01.287314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: warnings.warn(msg) Aug 23 08:17:01.287314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: Aug 23 08:17:01.288815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:01.403186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3c87e-2cd6-449f-814e-91fd01c62c23 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:01.408233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dee63-9e6f-4b3a-a99e-948c11a84649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:01.408917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dee63-9e6f-4b3a-a99e-948c11a84649 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:01.409199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dee63-9e6f-4b3a-a99e-948c11a84649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:01.409471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dee63-9e6f-4b3a-a99e-948c11a84649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:01.424260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168dee63-9e6f-4b3a-a99e-948c11a84649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:01.424260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168dee63-9e6f-4b3a-a99e-948c11a84649 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:01.433723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 829/1647] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:01.588423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd19b147e3690496992ca2a7233ce7768'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:01.595383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd19b147e3690496992ca2a7233ce7768'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:01.627097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd19b147e3690496992ca2a7233ce7768'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:01.629331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd19b147e3690496992ca2a7233ce7768'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:01.708115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd19b147e3690496992ca2a7233ce7768'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:01.766334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 819/1648] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:01 2022] DELETE /identity/v3/users/d19b147e3690496992ca2a7233ce7768 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:01.771319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6bdd-e969-4d35-9ca7-193865f11a45 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:01.771759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6bdd-e969-4d35-9ca7-193865f11a45 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:01.772491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6bdd-e969-4d35-9ca7-193865f11a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:01.824005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-78cb6bdd-e969-4d35-9ca7-193865f11a45 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:01.830576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-78cb6bdd-e969-4d35-9ca7-193865f11a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:01.898546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:02.039072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 830/1649] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:02.090749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:02.093169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:02.292270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d16be-5689-47e2-85f1-8b0bb9cd3bad admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:02.305680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3b660-0c93-4d55-bc96-f87c461ce958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:02.312284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b660-0c93-4d55-bc96-f87c461ce958 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:02.312735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b660-0c93-4d55-bc96-f87c461ce958 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:02.313022 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b660-0c93-4d55-bc96-f87c461ce958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:02.342235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3b660-0c93-4d55-bc96-f87c461ce958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:02.344976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3b660-0c93-4d55-bc96-f87c461ce958 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:02.376930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 820/1650] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:03.260467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb6bdd-e969-4d35-9ca7-193865f11a45 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:03.261995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-78cb6bdd-e969-4d35-9ca7-193865f11a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:03.284548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb6bdd-e969-4d35-9ca7-193865f11a45 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:03.303596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:03.304277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:03.304608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:03.304995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] PATH_INFO: `/v3/projects/c49d93c1beec4a329469f8f8d75d9af0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:03.307000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49d93c1beec4a329469f8f8d75d9af0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:03.308021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 08:17:03.308021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Aug 23 08:17:03.308021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: Aug 23 08:17:03.312392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:03.564359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49d93c1beec4a329469f8f8d75d9af0', 'request_id': 'req-9430a7de-d564-49a3-86c7-acff8ef15878'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:03.660464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49d93c1beec4a329469f8f8d75d9af0', 'request_id': 'req-9430a7de-d564-49a3-86c7-acff8ef15878'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:03.669049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49d93c1beec4a329469f8f8d75d9af0', 'request_id': 'req-9430a7de-d564-49a3-86c7-acff8ef15878'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:03.726771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] The token cache is being invalidate because project c49d93c1beec4a329469f8f8d75d9af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:03.727159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:03.728755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:03.736416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 831/1651] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:03 2022] DELETE /identity/v3/projects/c49d93c1beec4a329469f8f8d75d9af0 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:03.738163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be3012-ff83-4148-859a-20bb0f08677c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:03.738743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be3012-ff83-4148-859a-20bb0f08677c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:03.739210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be3012-ff83-4148-859a-20bb0f08677c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:03.759908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:03.761625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:03.867558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f1be3012-ff83-4148-859a-20bb0f08677c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:03.871343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f1be3012-ff83-4148-859a-20bb0f08677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:03.935880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:04.007333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a7de-d564-49a3-86c7-acff8ef15878 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:04.024278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:04.025665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:04.025948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:04.026221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] PATH_INFO: `/v3/projects/55ff9b7ff3b5483891849c10c99e2762` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:04.035433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] RBAC: Authorizing `identity:delete_project(project_id=55ff9b7ff3b5483891849c10c99e2762)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:04.037880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:04.123651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 821/1652] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:04.133046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0dabe-25c6-4aab-8c99-c405f55a694f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:04.133494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0dabe-25c6-4aab-8c99-c405f55a694f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:04.133928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0dabe-25c6-4aab-8c99-c405f55a694f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:04.165593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-eda0dabe-25c6-4aab-8c99-c405f55a694f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:04.167643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-eda0dabe-25c6-4aab-8c99-c405f55a694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:04.177318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 822/1653] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:17:04.181466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0dabe-25c6-4aab-8c99-c405f55a694f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:04.197125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0dabe-25c6-4aab-8c99-c405f55a694f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:04.202772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:04.203803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:04.204918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:04.205410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:04.206498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:04.207172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:04.209523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:04.226188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 823/1654] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:17:04.230350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:04.246360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:04.249974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2dc6-cf6e-49b4-b70e-ae32d7b20294 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:04.250535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55ff9b7ff3b5483891849c10c99e2762', 'request_id': 'req-e8f300db-5604-4175-a91c-b18fd83f556d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.355302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55ff9b7ff3b5483891849c10c99e2762', 'request_id': 'req-e8f300db-5604-4175-a91c-b18fd83f556d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.362513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55ff9b7ff3b5483891849c10c99e2762', 'request_id': 'req-e8f300db-5604-4175-a91c-b18fd83f556d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.399568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] The token cache is being invalidate because project 55ff9b7ff3b5483891849c10c99e2762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:04.400150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.401562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.406694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 832/1655] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:03 2022] DELETE /identity/v3/projects/55ff9b7ff3b5483891849c10c99e2762 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:04.411224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:04.413523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:04.414434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:04.415048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:04.421531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:04.421882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:04.421882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:04.421882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:04.441551 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f300db-5604-4175-a91c-b18fd83f556d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:04.454811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:04.456377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:04.456845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:04.457296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] PATH_INFO: `/v3/users/023015ede0b64d3bada5cb15a60b5efe` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:04.458673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=023015ede0b64d3bada5cb15a60b5efe)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:04.461187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:04.526677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 824/1656] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:04 2022] POST /identity/v3/projects => generated 371 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:04.530661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:04.533412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:04.673900 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf1b91-aea3-40b4-9823-461910df6eb8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:04.689952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:04.691108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:04.691593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:04.692399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:04.693485 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:04.702561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:04.708549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023015ede0b64d3bada5cb15a60b5efe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.725396 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023015ede0b64d3bada5cb15a60b5efe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.726383 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023015ede0b64d3bada5cb15a60b5efe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.727159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023015ede0b64d3bada5cb15a60b5efe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.759479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023015ede0b64d3bada5cb15a60b5efe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:04.862159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 825/1657] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:04 2022] POST /identity/v3/users => generated 286 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:04.866468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:04.870726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:04.923128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 833/1658] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:04 2022] DELETE /identity/v3/users/023015ede0b64d3bada5cb15a60b5efe => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:04.940975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f62af-dd76-4110-8920-aadcfc49c4ea admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:05.175601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf060-6247-4c02-9f3c-d4ad3726ea6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:05.176738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf060-6247-4c02-9f3c-d4ad3726ea6e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:05.177240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf060-6247-4c02-9f3c-d4ad3726ea6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:05.177871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf060-6247-4c02-9f3c-d4ad3726ea6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:05.178773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf060-6247-4c02-9f3c-d4ad3726ea6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:05.183848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcf060-6247-4c02-9f3c-d4ad3726ea6e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:05.221398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 826/1659] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:05.223228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:05.240388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a60a-1b2f-4893-aec0-aab68fd830d9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:05.265013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b0be4-2fa5-4a11-9773-8439fc52e5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:05.265811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b0be4-2fa5-4a11-9773-8439fc52e5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:05.267231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b0be4-2fa5-4a11-9773-8439fc52e5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:05.268953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b0be4-2fa5-4a11-9773-8439fc52e5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:05.269815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b0be4-2fa5-4a11-9773-8439fc52e5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:05.276331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b0be4-2fa5-4a11-9773-8439fc52e5e8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:05.299491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 834/1660] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:05.303097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf060-6247-4c02-9f3c-d4ad3726ea6e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:05.317781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf060-6247-4c02-9f3c-d4ad3726ea6e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:05.336258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4b626-e376-4892-9e4a-8057c481c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:05.337828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b626-e376-4892-9e4a-8057c481c145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:05.338122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b626-e376-4892-9e4a-8057c481c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:05.338403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b626-e376-4892-9e4a-8057c481c145 admin admin] PATH_INFO: `/v3/projects/67a05a96d9e84c33ac2b9efbaf7935ef/users/0fc5b2b720594d65b8711ab49f396d3f/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:05.375298 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4b626-e376-4892-9e4a-8057c481c145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67a05a96d9e84c33ac2b9efbaf7935ef, user_id=0fc5b2b720594d65b8711ab49f396d3f, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:05.383369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4b626-e376-4892-9e4a-8057c481c145 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:05.503675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1b998-7be6-41f3-aa46-c07680d551e0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:05.504601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1b998-7be6-41f3-aa46-c07680d551e0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:05.505110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1b998-7be6-41f3-aa46-c07680d551e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:05.539192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 827/1661] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:05 2022] PUT /identity/v3/projects/67a05a96d9e84c33ac2b9efbaf7935ef/users/0fc5b2b720594d65b8711ab49f396d3f/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:05.541869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde42074-95ba-4b5a-8637-01800c4d7498 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:05.542173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde42074-95ba-4b5a-8637-01800c4d7498 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:05.545346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde42074-95ba-4b5a-8637-01800c4d7498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:05.546557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b6b1b998-7be6-41f3-aa46-c07680d551e0 None None] MFA Rules not processed for user `b3c984088c0448f8abc0a0b60b820b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:05.549012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b6b1b998-7be6-41f3-aa46-c07680d551e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:05.612426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bde42074-95ba-4b5a-8637-01800c4d7498 None None] MFA Rules not processed for user `0fc5b2b720594d65b8711ab49f396d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:05.613778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bde42074-95ba-4b5a-8637-01800c4d7498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:05.687277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 835/1662] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:05.689778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97fab6-c3f0-4d74-aeec-75f019abe0cf None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:05.690610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97fab6-c3f0-4d74-aeec-75f019abe0cf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:05.691072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97fab6-c3f0-4d74-aeec-75f019abe0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:05.720437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6e97fab6-c3f0-4d74-aeec-75f019abe0cf None None] MFA Rules not processed for user `b3c984088c0448f8abc0a0b60b820b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:05.722510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6e97fab6-c3f0-4d74-aeec-75f019abe0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:05.731932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 836/1663] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:05.749601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97fab6-c3f0-4d74-aeec-75f019abe0cf None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:05.752403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6e97fab6-c3f0-4d74-aeec-75f019abe0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:05.798874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 828/1664] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:05.803266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde42074-95ba-4b5a-8637-01800c4d7498 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:05.941087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97fab6-c3f0-4d74-aeec-75f019abe0cf None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:05.960868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde42074-95ba-4b5a-8637-01800c4d7498 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:05.961609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:05.962455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:05.962845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:05.963283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:05.974859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71e42f-dd95-4f6c-870b-2210f17f1b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:05.975912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71e42f-dd95-4f6c-870b-2210f17f1b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:05.976352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71e42f-dd95-4f6c-870b-2210f17f1b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:05.977067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71e42f-dd95-4f6c-870b-2210f17f1b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:05.977847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71e42f-dd95-4f6c-870b-2210f17f1b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:05.979337 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:05.980297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71e42f-dd95-4f6c-870b-2210f17f1b3e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:05.981861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:05.997617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 837/1665] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:05.997762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:06.176953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:06.178495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:06.179414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 829/1666] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:05 2022] POST /identity/v3/projects => generated 371 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:06.183247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71e42f-dd95-4f6c-870b-2210f17f1b3e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:06.233158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9bbd5-ef34-4fa0-af96-fa2c51ab9670 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:06.266339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:06.310329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71e42f-dd95-4f6c-870b-2210f17f1b3e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:06.324833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e070d-f3a6-4f7f-b2c6-1d3fb9dbb2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:06.326857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e070d-f3a6-4f7f-b2c6-1d3fb9dbb2ea admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:06.327341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e070d-f3a6-4f7f-b2c6-1d3fb9dbb2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:06.327840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e070d-f3a6-4f7f-b2c6-1d3fb9dbb2ea admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:06.329172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e070d-f3a6-4f7f-b2c6-1d3fb9dbb2ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:06.331827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e070d-f3a6-4f7f-b2c6-1d3fb9dbb2ea admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:06.429273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:06.432296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:06.432296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:06.432296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] PATH_INFO: `/v3/projects/2655779c0d4d465e8b7b3452ce95aabd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:06.433511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2655779c0d4d465e8b7b3452ce95aabd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:06.435961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:06.466390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 830/1667] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:06 2022] POST /identity/v3/users => generated 292 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:06.468096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e070d-f3a6-4f7f-b2c6-1d3fb9dbb2ea admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:06.483931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e070d-f3a6-4f7f-b2c6-1d3fb9dbb2ea admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:06.503778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328060bd-b71f-458e-8d76-0d665fcc1942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:06.503778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328060bd-b71f-458e-8d76-0d665fcc1942 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:06.503778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328060bd-b71f-458e-8d76-0d665fcc1942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:06.503778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328060bd-b71f-458e-8d76-0d665fcc1942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:06.504143 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328060bd-b71f-458e-8d76-0d665fcc1942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:06.506798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328060bd-b71f-458e-8d76-0d665fcc1942 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:06.558633 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 831/1668] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:06.560027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328060bd-b71f-458e-8d76-0d665fcc1942 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:06.584621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328060bd-b71f-458e-8d76-0d665fcc1942 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:06.594026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43566d38-c1ce-4885-8e98-3a44ce5081ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:06.594026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43566d38-c1ce-4885-8e98-3a44ce5081ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:06.597053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43566d38-c1ce-4885-8e98-3a44ce5081ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:06.597609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43566d38-c1ce-4885-8e98-3a44ce5081ac admin admin] PATH_INFO: `/v3/projects/ea826bc8900f4d0a95c5144384c39184/users/deadeb680ae948b999a6b37127ea2750/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:06.656652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43566d38-c1ce-4885-8e98-3a44ce5081ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea826bc8900f4d0a95c5144384c39184, user_id=deadeb680ae948b999a6b37127ea2750, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:06.663740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2655779c0d4d465e8b7b3452ce95aabd', 'request_id': 'req-abc75983-8731-49e7-9770-3aa8d8af1375'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:06.667414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43566d38-c1ce-4885-8e98-3a44ce5081ac admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:06.766688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2655779c0d4d465e8b7b3452ce95aabd', 'request_id': 'req-abc75983-8731-49e7-9770-3aa8d8af1375'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:06.788726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2655779c0d4d465e8b7b3452ce95aabd', 'request_id': 'req-abc75983-8731-49e7-9770-3aa8d8af1375'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:06.803475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 832/1669] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:06 2022] PUT /identity/v3/projects/ea826bc8900f4d0a95c5144384c39184/users/deadeb680ae948b999a6b37127ea2750/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:06.807563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43566d38-c1ce-4885-8e98-3a44ce5081ac admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:06.836972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] The token cache is being invalidate because project 2655779c0d4d465e8b7b3452ce95aabd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:06.837638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:06.839782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-abc75983-8731-49e7-9770-3aa8d8af1375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:06.846241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 838/1670] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:06 2022] DELETE /identity/v3/projects/2655779c0d4d465e8b7b3452ce95aabd => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:06.850903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d697732-b53f-41c7-9f12-8d256447ca02 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:06.851403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d697732-b53f-41c7-9f12-8d256447ca02 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:06.855017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d697732-b53f-41c7-9f12-8d256447ca02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:06.900925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9d697732-b53f-41c7-9f12-8d256447ca02 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:06.911256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9d697732-b53f-41c7-9f12-8d256447ca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:07.090593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 839/1671] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:07.110128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43566d38-c1ce-4885-8e98-3a44ce5081ac admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:07.118481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:07.120209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc016-a0b3-4df0-824c-7517301681df None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.120600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc016-a0b3-4df0-824c-7517301681df None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.120912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddc016-a0b3-4df0-824c-7517301681df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.132541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:07.133469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.133932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.134385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.135345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:07.142156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:07.156846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 833/1672] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:07.160948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:07.165836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:07.198271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f3ddc016-a0b3-4df0-824c-7517301681df None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:07.200905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b7664-b4b2-442b-9ede-c068ebee829f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:07.202278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f3ddc016-a0b3-4df0-824c-7517301681df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:07.214581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e2169-7f75-4588-8af4-3f06b4f1f7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:07.215626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2169-7f75-4588-8af4-3f06b4f1f7af admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.216301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2169-7f75-4588-8af4-3f06b4f1f7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.216816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2169-7f75-4588-8af4-3f06b4f1f7af admin admin] PATH_INFO: `/v3/projects/ea826bc8900f4d0a95c5144384c39184/users/deadeb680ae948b999a6b37127ea2750/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.222617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e2169-7f75-4588-8af4-3f06b4f1f7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea826bc8900f4d0a95c5144384c39184, user_id=deadeb680ae948b999a6b37127ea2750, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:07.228283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e2169-7f75-4588-8af4-3f06b4f1f7af admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:07.242133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 840/1673] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:07.245139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddc016-a0b3-4df0-824c-7517301681df None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:07.256399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 834/1674] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:07 2022] PUT /identity/v3/projects/ea826bc8900f4d0a95c5144384c39184/users/deadeb680ae948b999a6b37127ea2750/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:07.273693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b880e-c649-4b8f-97f2-afaf1f97049d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.274100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b880e-c649-4b8f-97f2-afaf1f97049d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.274732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b880e-c649-4b8f-97f2-afaf1f97049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.281463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddc016-a0b3-4df0-824c-7517301681df None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:07.340698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-362b880e-c649-4b8f-97f2-afaf1f97049d None None] MFA Rules not processed for user `deadeb680ae948b999a6b37127ea2750`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:07.342096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-362b880e-c649-4b8f-97f2-afaf1f97049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:07.447071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6c4d3-5ba9-4708-accd-7fc5bcf8ee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:07.447706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6c4d3-5ba9-4708-accd-7fc5bcf8ee62 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.448185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6c4d3-5ba9-4708-accd-7fc5bcf8ee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.448456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6c4d3-5ba9-4708-accd-7fc5bcf8ee62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.449304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6c4d3-5ba9-4708-accd-7fc5bcf8ee62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:07.455368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6c4d3-5ba9-4708-accd-7fc5bcf8ee62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:07.457136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6c4d3-5ba9-4708-accd-7fc5bcf8ee62 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:07.461328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 835/1675] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:07.462956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df022c7-17e6-48d3-8b2a-5ce81eb38e11 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.463291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df022c7-17e6-48d3-8b2a-5ce81eb38e11 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.463571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df022c7-17e6-48d3-8b2a-5ce81eb38e11 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.464942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 836/1676] 10.222.0.58 () {54 vars in 912 bytes} [Tue Aug 23 08:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:17:07.469189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df022c7-17e6-48d3-8b2a-5ce81eb38e11 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:07.471441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3df022c7-17e6-48d3-8b2a-5ce81eb38e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:07.490494 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 841/1677] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:17:07.491969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdf1a-7adc-4407-8a6b-37e306075a54 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.492245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdf1a-7adc-4407-8a6b-37e306075a54 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.492514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cdf1a-7adc-4407-8a6b-37e306075a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.561123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df022c7-17e6-48d3-8b2a-5ce81eb38e11 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:07.582903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ee77f-a85d-4166-9a75-1fc7732e2333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:07.583777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ee77f-a85d-4166-9a75-1fc7732e2333 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.584139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ee77f-a85d-4166-9a75-1fc7732e2333 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.584482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ee77f-a85d-4166-9a75-1fc7732e2333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.584603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bc9cdf1a-7adc-4407-8a6b-37e306075a54 None None] MFA Rules not processed for user `d99632164b3b4a1f95736a65da217db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:07.586430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bc9cdf1a-7adc-4407-8a6b-37e306075a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:07.596185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ee77f-a85d-4166-9a75-1fc7732e2333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:07.598999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ee77f-a85d-4166-9a75-1fc7732e2333 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:07.627088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 837/1678] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:07.629546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ee77f-a85d-4166-9a75-1fc7732e2333 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:07.655464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ee77f-a85d-4166-9a75-1fc7732e2333 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:07.661843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7ba86-c5b5-4f4f-9c8c-03d0aa0c303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:07.662808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7ba86-c5b5-4f4f-9c8c-03d0aa0c303e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.663289 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7ba86-c5b5-4f4f-9c8c-03d0aa0c303e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.663899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7ba86-c5b5-4f4f-9c8c-03d0aa0c303e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.664781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7ba86-c5b5-4f4f-9c8c-03d0aa0c303e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:07.667016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7ba86-c5b5-4f4f-9c8c-03d0aa0c303e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:07.701564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 842/1679] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:17:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:07.716980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cdf1a-7adc-4407-8a6b-37e306075a54 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:07.718618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bc9cdf1a-7adc-4407-8a6b-37e306075a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:07.736308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cdf1a-7adc-4407-8a6b-37e306075a54 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:07.761870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dddf3-58b8-4566-9a93-aafb14226df4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:07.761870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dddf3-58b8-4566-9a93-aafb14226df4 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:07.761870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dddf3-58b8-4566-9a93-aafb14226df4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:07.761870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dddf3-58b8-4566-9a93-aafb14226df4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:07.772118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dddf3-58b8-4566-9a93-aafb14226df4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:07.774844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dddf3-58b8-4566-9a93-aafb14226df4 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:07.794203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 843/1680] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:17:07.870784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 838/1681] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:07 2022] POST /identity/v3/projects => generated 379 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:07.871977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dddf3-58b8-4566-9a93-aafb14226df4 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:07.899433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dddf3-58b8-4566-9a93-aafb14226df4 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:08.042827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7ba86-c5b5-4f4f-9c8c-03d0aa0c303e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:08.051316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746878de-8d25-44f5-a0a1-beb3abf94bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:08.052199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746878de-8d25-44f5-a0a1-beb3abf94bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:08.053977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:08.054700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746878de-8d25-44f5-a0a1-beb3abf94bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:08.055080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746878de-8d25-44f5-a0a1-beb3abf94bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:08.055857 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746878de-8d25-44f5-a0a1-beb3abf94bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:08.059239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746878de-8d25-44f5-a0a1-beb3abf94bd3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:08.135543 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:08.199645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 844/1682] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:07 2022] POST /identity/v3/users => generated 290 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:08.203241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746878de-8d25-44f5-a0a1-beb3abf94bd3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:08.227749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746878de-8d25-44f5-a0a1-beb3abf94bd3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:08.236601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7ba86-c5b5-4f4f-9c8c-03d0aa0c303e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:08.237475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a506-422a-461b-927f-add5fca17a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:08.238195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a506-422a-461b-927f-add5fca17a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:08.238544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a506-422a-461b-927f-add5fca17a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:08.238892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a506-422a-461b-927f-add5fca17a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:08.240815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77a506-422a-461b-927f-add5fca17a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:08.243033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b565b1-a430-42f5-b9db-c256f2c0bb92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:08.243181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b565b1-a430-42f5-b9db-c256f2c0bb92 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:08.243631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b565b1-a430-42f5-b9db-c256f2c0bb92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:08.243942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b565b1-a430-42f5-b9db-c256f2c0bb92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:08.244541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77a506-422a-461b-927f-add5fca17a40 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:08.247063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-94b565b1-a430-42f5-b9db-c256f2c0bb92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:08.298852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 845/1683] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:08.300642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a506-422a-461b-927f-add5fca17a40 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:08.342135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a506-422a-461b-927f-add5fca17a40 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:08.356474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4dd23-38b1-4c51-b176-ff668311b552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:08.357204 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4dd23-38b1-4c51-b176-ff668311b552 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:08.358392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4dd23-38b1-4c51-b176-ff668311b552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:08.362022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4dd23-38b1-4c51-b176-ff668311b552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:08.362964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba4dd23-38b1-4c51-b176-ff668311b552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:08.364781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba4dd23-38b1-4c51-b176-ff668311b552 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:08.394831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b565b1-a430-42f5-b9db-c256f2c0bb92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:08.408386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b565b1-a430-42f5-b9db-c256f2c0bb92 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:08.412056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 846/1684] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:08.422612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4dd23-38b1-4c51-b176-ff668311b552 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:08.436903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 839/1685] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:08.456969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4dd23-38b1-4c51-b176-ff668311b552 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:08.476418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc0668-c8c0-4d0d-bca6-7b9a0343dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:08.477420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc0668-c8c0-4d0d-bca6-7b9a0343dca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:08.477420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc0668-c8c0-4d0d-bca6-7b9a0343dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:08.477851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc0668-c8c0-4d0d-bca6-7b9a0343dca3 admin admin] PATH_INFO: `/v3/projects/07de55522b6549b6be07de3f5d64d03f/users/66c16aa2302143aa837abb2bafae4ca9/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:08.516243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc0668-c8c0-4d0d-bca6-7b9a0343dca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07de55522b6549b6be07de3f5d64d03f, user_id=66c16aa2302143aa837abb2bafae4ca9, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:08.520688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc0668-c8c0-4d0d-bca6-7b9a0343dca3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:08.647112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 847/1686] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:08 2022] PUT /identity/v3/projects/07de55522b6549b6be07de3f5d64d03f/users/66c16aa2302143aa837abb2bafae4ca9/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:08.649300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c954bfa-754c-4b50-887a-e2c78aef7ab3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:08.649482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c954bfa-754c-4b50-887a-e2c78aef7ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:08.650137 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c954bfa-754c-4b50-887a-e2c78aef7ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:08.712113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5c954bfa-754c-4b50-887a-e2c78aef7ab3 None None] MFA Rules not processed for user `66c16aa2302143aa837abb2bafae4ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:08.713982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5c954bfa-754c-4b50-887a-e2c78aef7ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:08.764890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b2adc-fe53-42c6-af3a-10940024b619 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:08.764890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b2adc-fe53-42c6-af3a-10940024b619 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:08.764890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b2adc-fe53-42c6-af3a-10940024b619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:08.844945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-537b2adc-fe53-42c6-af3a-10940024b619 None None] MFA Rules not processed for user `dd81f4241d7343e990aea15e49be5f81`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:08.847286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-537b2adc-fe53-42c6-af3a-10940024b619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:08.860294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:08.939787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 840/1687] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:08.944981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c954bfa-754c-4b50-887a-e2c78aef7ab3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:09.082530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c954bfa-754c-4b50-887a-e2c78aef7ab3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:09.104294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 848/1688] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:09.104294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5cc1d-b171-41e6-95f6-443137afe262 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.104294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5cc1d-b171-41e6-95f6-443137afe262 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.104294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5cc1d-b171-41e6-95f6-443137afe262 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.106405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 849/1689] 10.222.0.58 () {54 vars in 961 bytes} [Tue Aug 23 08:17:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:17:09.107327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5cc1d-b171-41e6-95f6-443137afe262 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:09.111516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dcd5cc1d-b171-41e6-95f6-443137afe262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:09.116080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897400cf-a0ac-4a06-8965-88a1ab11dec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:09.116590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897400cf-a0ac-4a06-8965-88a1ab11dec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.116868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897400cf-a0ac-4a06-8965-88a1ab11dec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.117142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897400cf-a0ac-4a06-8965-88a1ab11dec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.119893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897400cf-a0ac-4a06-8965-88a1ab11dec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:09.134039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897400cf-a0ac-4a06-8965-88a1ab11dec3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:09.152069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5cc1d-b171-41e6-95f6-443137afe262 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:09.162748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:09.164144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.164692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.164899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] PATH_INFO: `/v3/users/e1ad75433eca4c3e8ec6cbc6fa5cc79e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.166219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ad75433eca4c3e8ec6cbc6fa5cc79e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:09.180397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:09.304575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 841/1690] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:08 2022] POST /identity/v3/projects => generated 381 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:09.306154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ad7a-064b-4b1e-a66c-bd1f8ff784e3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.306449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ad7a-064b-4b1e-a66c-bd1f8ff784e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.306724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ad7a-064b-4b1e-a66c-bd1f8ff784e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.320642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:09.337876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-dc45ad7a-064b-4b1e-a66c-bd1f8ff784e3 None None] MFA Rules not processed for user `ce316aa5c0b54ac99ade6cc4163ee209`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:09.339229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-dc45ad7a-064b-4b1e-a66c-bd1f8ff784e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:09.427729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 842/1691] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:09.429135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ab679-c4e8-4453-a0da-1c4d012ddd0f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.429421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ab679-c4e8-4453-a0da-1c4d012ddd0f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.429688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ab679-c4e8-4453-a0da-1c4d012ddd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.464476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b13ab679-c4e8-4453-a0da-1c4d012ddd0f None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:09.466905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b13ab679-c4e8-4453-a0da-1c4d012ddd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:09.480951 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ad75433eca4c3e8ec6cbc6fa5cc79e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:09.510223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ad75433eca4c3e8ec6cbc6fa5cc79e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:09.515226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ad75433eca4c3e8ec6cbc6fa5cc79e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:09.516603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ad75433eca4c3e8ec6cbc6fa5cc79e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:09.555630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48d59a6d-3f82-4d80-99f8-69e2ed50645e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ad75433eca4c3e8ec6cbc6fa5cc79e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:09.617756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 843/1692] 10.222.0.58 () {58 vars in 1058 bytes} [Tue Aug 23 08:17:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:09.627940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f851a-5ee4-490b-91de-707c23de1fac None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.628287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f851a-5ee4-490b-91de-707c23de1fac None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.628512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f851a-5ee4-490b-91de-707c23de1fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.644759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 850/1693] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:09 2022] DELETE /identity/v3/users/e1ad75433eca4c3e8ec6cbc6fa5cc79e => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:09.647078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f7646-d539-42a4-af36-cb875141ee2a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.647897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f7646-d539-42a4-af36-cb875141ee2a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.648404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f7646-d539-42a4-af36-cb875141ee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.658383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0b4f851a-5ee4-490b-91de-707c23de1fac None None] MFA Rules not processed for user `dd81f4241d7343e990aea15e49be5f81`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:09.659513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0b4f851a-5ee4-490b-91de-707c23de1fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:09.672276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-349f7646-d539-42a4-af36-cb875141ee2a None None] MFA Rules not processed for user `b3c984088c0448f8abc0a0b60b820b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:09.678624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-349f7646-d539-42a4-af36-cb875141ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:09.738640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 844/1694] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:09.739810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f851a-5ee4-490b-91de-707c23de1fac None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:09.741291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0b4f851a-5ee4-490b-91de-707c23de1fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:09.777627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 851/1695] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:09.777627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f7646-d539-42a4-af36-cb875141ee2a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:09.777627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-349f7646-d539-42a4-af36-cb875141ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:09.849878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f851a-5ee4-490b-91de-707c23de1fac None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:09.859724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212b8ed-4c6b-4c68-93af-ec200325ce32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:09.860723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b8ed-4c6b-4c68-93af-ec200325ce32 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.861211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b8ed-4c6b-4c68-93af-ec200325ce32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.861708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b8ed-4c6b-4c68-93af-ec200325ce32 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.862849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212b8ed-4c6b-4c68-93af-ec200325ce32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:09.865150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212b8ed-4c6b-4c68-93af-ec200325ce32 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:09.871215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f7646-d539-42a4-af36-cb875141ee2a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:09.881677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:09.882534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:09.883020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:09.883545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:09.886224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:09.924924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:09.961566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:09.963361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:09.974420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 852/1696] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:09.975389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:10.000401 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 845/1697] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:09 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:10.001879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d4b7-7724-4a3a-8756-4734364f20e3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.002168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d4b7-7724-4a3a-8756-4734364f20e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.002507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d4b7-7724-4a3a-8756-4734364f20e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.045205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:10.057003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b617d4b7-7724-4a3a-8756-4734364f20e3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:10.061816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b617d4b7-7724-4a3a-8756-4734364f20e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:10.066101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a5a1f-13a3-4b98-be3c-17903d7ce6bd service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:10.072296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:10.072917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.073584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.073773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.075838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:10.076219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 846/1698] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:10.077090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617d4b7-7724-4a3a-8756-4734364f20e3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:10.078870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b617d4b7-7724-4a3a-8756-4734364f20e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:10.094466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:10.096827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:10.108491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617d4b7-7724-4a3a-8756-4734364f20e3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:10.109700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 853/1699] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:10.110717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:10.130198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163cdd7-d747-4aef-b054-bd0687a4140b service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:10.132194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:10.133738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.134075 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.134397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.135396 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:10.136006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.136284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.136555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.137195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:10.137708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:10.139512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:10.156585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 854/1700] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:10.157695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:10.159555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:10.172594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:10.172594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:10.233878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 847/1701] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:10.241044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:10.261830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a32a4f-0337-44f1-9672-70af52f0f5df admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:10.266391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f3bb3-bd53-49ba-ba11-7cb97c8881d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:10.267144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f3bb3-bd53-49ba-ba11-7cb97c8881d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.267389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f3bb3-bd53-49ba-ba11-7cb97c8881d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.267740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f3bb3-bd53-49ba-ba11-7cb97c8881d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.284594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53018-0c75-4ac9-849f-5f39aef68b2d service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:10.287741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f3bb3-bd53-49ba-ba11-7cb97c8881d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:10.289213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f3bb3-bd53-49ba-ba11-7cb97c8881d9 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:10.330534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec835f-eb42-45a7-98d3-0a7a74a021a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:10.331256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec835f-eb42-45a7-98d3-0a7a74a021a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.331537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec835f-eb42-45a7-98d3-0a7a74a021a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.331806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec835f-eb42-45a7-98d3-0a7a74a021a9 admin admin] PATH_INFO: `/v3/projects/6dd180c7801143f39fa02c169b07ae21/users/ff892413e4b94c9a849869ca4fd9d976/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.356161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 855/1702] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:10.384084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec835f-eb42-45a7-98d3-0a7a74a021a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd180c7801143f39fa02c169b07ae21, user_id=ff892413e4b94c9a849869ca4fd9d976, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:10.392990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec835f-eb42-45a7-98d3-0a7a74a021a9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:10.511505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 848/1703] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:10 2022] PUT /identity/v3/projects/6dd180c7801143f39fa02c169b07ae21/users/ff892413e4b94c9a849869ca4fd9d976/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:10.512912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f3bb3-bd53-49ba-ba11-7cb97c8881d9 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:10.527058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f3bb3-bd53-49ba-ba11-7cb97c8881d9 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:10.732795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8df5e-6856-437e-9f31-6d2788626a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:10.736845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8df5e-6856-437e-9f31-6d2788626a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.736845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8df5e-6856-437e-9f31-6d2788626a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.736845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8df5e-6856-437e-9f31-6d2788626a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.736845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8df5e-6856-437e-9f31-6d2788626a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:10.736845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8df5e-6856-437e-9f31-6d2788626a15 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:10.768966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 856/1704] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:10.774412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec835f-eb42-45a7-98d3-0a7a74a021a9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:10.800457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec835f-eb42-45a7-98d3-0a7a74a021a9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:10.807758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975140c3-3c86-4361-89db-28839c4b6332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:10.808445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975140c3-3c86-4361-89db-28839c4b6332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.808729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975140c3-3c86-4361-89db-28839c4b6332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.809007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975140c3-3c86-4361-89db-28839c4b6332 admin admin] PATH_INFO: `/v3/projects/6dd180c7801143f39fa02c169b07ae21/users/ff892413e4b94c9a849869ca4fd9d976/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.815500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975140c3-3c86-4361-89db-28839c4b6332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd180c7801143f39fa02c169b07ae21, user_id=ff892413e4b94c9a849869ca4fd9d976, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:10.818465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975140c3-3c86-4361-89db-28839c4b6332 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:10.851805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 849/1705] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:10 2022] PUT /identity/v3/projects/6dd180c7801143f39fa02c169b07ae21/users/ff892413e4b94c9a849869ca4fd9d976/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:10.865055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c76a-62f1-491e-9ea9-791857638125 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.868276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c76a-62f1-491e-9ea9-791857638125 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.868276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c76a-62f1-491e-9ea9-791857638125 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.868276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 857/1706] 10.222.0.58 () {54 vars in 912 bytes} [Tue Aug 23 08:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:17:10.874328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956b99c-3e2f-4425-926b-8ad8da1de54d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.874742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956b99c-3e2f-4425-926b-8ad8da1de54d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.875182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956b99c-3e2f-4425-926b-8ad8da1de54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.878015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42594b-1ad5-46d8-8110-828edebd7012 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:10.878414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42594b-1ad5-46d8-8110-828edebd7012 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:10.879536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42594b-1ad5-46d8-8110-828edebd7012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:10.933577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e956b99c-3e2f-4425-926b-8ad8da1de54d None None] MFA Rules not processed for user `d99632164b3b4a1f95736a65da217db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:10.934760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e956b99c-3e2f-4425-926b-8ad8da1de54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:10.954099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bc42594b-1ad5-46d8-8110-828edebd7012 None None] MFA Rules not processed for user `ff892413e4b94c9a849869ca4fd9d976`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:10.955795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bc42594b-1ad5-46d8-8110-828edebd7012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:11.086662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 858/1707] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:17:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:11.101177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956b99c-3e2f-4425-926b-8ad8da1de54d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:11.102863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e956b99c-3e2f-4425-926b-8ad8da1de54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:11.125692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956b99c-3e2f-4425-926b-8ad8da1de54d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:11.129866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 850/1708] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:11.139998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:11.140638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:11.140918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:11.141193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:11.168528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:11.171167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:11.198033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 859/1709] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:17:11.252846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42594b-1ad5-46d8-8110-828edebd7012 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:11.254908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bc42594b-1ad5-46d8-8110-828edebd7012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:11.302882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:11.304625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:11.499581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42594b-1ad5-46d8-8110-828edebd7012 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:11.506647 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2983a9f-837e-44fe-93d3-aa0ad23ab099 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:11.518125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090bd10-91bc-409d-958f-878dc3eaa5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:11.523472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:11.525076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090bd10-91bc-409d-958f-878dc3eaa5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:11.525421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090bd10-91bc-409d-958f-878dc3eaa5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:11.526679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:11.527550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090bd10-91bc-409d-958f-878dc3eaa5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:11.528861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:11.528861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] PATH_INFO: `/v3/projects/df0b6e7c14ba4a4eba256ad043f52c5d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:11.534415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0b6e7c14ba4a4eba256ad043f52c5d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:11.544752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:11.605288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090bd10-91bc-409d-958f-878dc3eaa5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:11.621821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090bd10-91bc-409d-958f-878dc3eaa5f0 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:11.684024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 851/1710] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:11.833663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0b6e7c14ba4a4eba256ad043f52c5d', 'request_id': 'req-5fcca72b-875b-4eea-a227-510bc105bfe1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:11.995040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0b6e7c14ba4a4eba256ad043f52c5d', 'request_id': 'req-5fcca72b-875b-4eea-a227-510bc105bfe1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:12.008920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0b6e7c14ba4a4eba256ad043f52c5d', 'request_id': 'req-5fcca72b-875b-4eea-a227-510bc105bfe1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:12.046488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] The token cache is being invalidate because project df0b6e7c14ba4a4eba256ad043f52c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:12.046996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:12.054875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5fcca72b-875b-4eea-a227-510bc105bfe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:12.054875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 860/1711] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:11 2022] DELETE /identity/v3/projects/df0b6e7c14ba4a4eba256ad043f52c5d => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:12.059623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1e27-353d-4523-b7a3-bc1dc2da30e0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:12.060041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1e27-353d-4523-b7a3-bc1dc2da30e0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:12.060435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d1e27-353d-4523-b7a3-bc1dc2da30e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:12.125882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0e6d1e27-353d-4523-b7a3-bc1dc2da30e0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:12.129276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0e6d1e27-353d-4523-b7a3-bc1dc2da30e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:12.139422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:12.368039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 852/1712] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:12.374173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcd0d1-172d-497b-b0a7-90bdf561391b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:12.375603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcd0d1-172d-497b-b0a7-90bdf561391b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:12.375896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcd0d1-172d-497b-b0a7-90bdf561391b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:12.477404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fabcd0d1-172d-497b-b0a7-90bdf561391b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:12.481609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fabcd0d1-172d-497b-b0a7-90bdf561391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:12.506507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d1e27-353d-4523-b7a3-bc1dc2da30e0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:12.520819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d1e27-353d-4523-b7a3-bc1dc2da30e0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:12.522570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 861/1713] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:12.545074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:12.545705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:12.545990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:12.546262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:12.546910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:12.562969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:12.564811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:12.600298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcd0d1-172d-497b-b0a7-90bdf561391b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:12.625395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcd0d1-172d-497b-b0a7-90bdf561391b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:12.631044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 853/1714] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:17:12.693722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:12.695047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:12.695333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:12.695606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:12.696245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:12.698479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:12.861042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:12.901373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:12.901597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 862/1715] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:12 2022] POST /identity/v3/projects => generated 367 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:12.921396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753fdf1-0f5d-4104-aa95-c8721cf75bca admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:13.211730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57765b-0e47-460c-99c3-698a01b7b8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:13.212615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57765b-0e47-460c-99c3-698a01b7b8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:13.217551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57765b-0e47-460c-99c3-698a01b7b8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:13.217953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57765b-0e47-460c-99c3-698a01b7b8c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:13.221210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57765b-0e47-460c-99c3-698a01b7b8c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:13.232910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57765b-0e47-460c-99c3-698a01b7b8c2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:13.334840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:13.337226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:13.370356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895766-404a-416e-9423-6f233f78ab58 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:13.385375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:13.387507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:13.387911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:13.388301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] PATH_INFO: `/v3/users/66c16aa2302143aa837abb2bafae4ca9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:13.394054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66c16aa2302143aa837abb2bafae4ca9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:13.396990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:13.413551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 854/1716] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:12 2022] POST /identity/v3/users => generated 284 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:13.415717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57765b-0e47-460c-99c3-698a01b7b8c2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:13.437046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57765b-0e47-460c-99c3-698a01b7b8c2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:13.447906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc7a28-384c-4298-8131-365f8cc6770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:13.448856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7a28-384c-4298-8131-365f8cc6770b admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:13.449366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7a28-384c-4298-8131-365f8cc6770b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:13.452567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc7a28-384c-4298-8131-365f8cc6770b admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:13.453395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc7a28-384c-4298-8131-365f8cc6770b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:13.457099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc7a28-384c-4298-8131-365f8cc6770b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:13.472073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 855/1717] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:13.480072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc7a28-384c-4298-8131-365f8cc6770b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:13.496566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc7a28-384c-4298-8131-365f8cc6770b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:13.501548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38895b-64a3-4d2d-8562-f81af520e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:13.506683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38895b-64a3-4d2d-8562-f81af520e0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:13.506683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38895b-64a3-4d2d-8562-f81af520e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:13.506683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38895b-64a3-4d2d-8562-f81af520e0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:13.506683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38895b-64a3-4d2d-8562-f81af520e0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:13.506683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38895b-64a3-4d2d-8562-f81af520e0c0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:13.526130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 856/1718] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:13.531393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38895b-64a3-4d2d-8562-f81af520e0c0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:13.542393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38895b-64a3-4d2d-8562-f81af520e0c0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:13.547844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32953414-63a9-4c37-96ca-32a091064823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:13.549382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32953414-63a9-4c37-96ca-32a091064823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:13.549980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32953414-63a9-4c37-96ca-32a091064823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:13.550558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32953414-63a9-4c37-96ca-32a091064823 admin admin] PATH_INFO: `/v3/projects/d5a0727ae3364629a8cdd35618e99f20/users/d263d7c1c7004ed7b26a0bc38bd98bdb/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:13.619919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32953414-63a9-4c37-96ca-32a091064823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a0727ae3364629a8cdd35618e99f20, user_id=d263d7c1c7004ed7b26a0bc38bd98bdb, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:13.623751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32953414-63a9-4c37-96ca-32a091064823 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:13.710897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66c16aa2302143aa837abb2bafae4ca9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:13.722087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66c16aa2302143aa837abb2bafae4ca9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:13.723297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66c16aa2302143aa837abb2bafae4ca9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:13.724239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66c16aa2302143aa837abb2bafae4ca9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:13.742731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bc7865aa-973f-4f33-ba1d-3c5f73ab2292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66c16aa2302143aa837abb2bafae4ca9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:13.774688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 857/1719] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:13 2022] PUT /identity/v3/projects/d5a0727ae3364629a8cdd35618e99f20/users/d263d7c1c7004ed7b26a0bc38bd98bdb/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:13.786536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a49db-b0b9-43ec-b655-76163a94383e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:13.786820 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a49db-b0b9-43ec-b655-76163a94383e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:13.787124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a49db-b0b9-43ec-b655-76163a94383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:13.846985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-784a49db-b0b9-43ec-b655-76163a94383e None None] MFA Rules not processed for user `d263d7c1c7004ed7b26a0bc38bd98bdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:13.848515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-784a49db-b0b9-43ec-b655-76163a94383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:13.868339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 863/1720] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:13 2022] DELETE /identity/v3/users/66c16aa2302143aa837abb2bafae4ca9 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:13.875300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ce4b-1a0c-4054-82eb-57c7b759d32e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:13.875744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ce4b-1a0c-4054-82eb-57c7b759d32e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:13.876219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ce4b-1a0c-4054-82eb-57c7b759d32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:13.924272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5480ce4b-1a0c-4054-82eb-57c7b759d32e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:13.926804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5480ce4b-1a0c-4054-82eb-57c7b759d32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:14.019781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 858/1721] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:14.034353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a49db-b0b9-43ec-b655-76163a94383e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:14.036786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-784a49db-b0b9-43ec-b655-76163a94383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:14.056269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 864/1722] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:14.102242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480ce4b-1a0c-4054-82eb-57c7b759d32e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:14.108305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5480ce4b-1a0c-4054-82eb-57c7b759d32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:14.120134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a49db-b0b9-43ec-b655-76163a94383e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:14.187077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:14.188119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:14.188545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:14.188990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:14.260969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:14.262712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:14.300619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480ce4b-1a0c-4054-82eb-57c7b759d32e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:14.320261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 859/1723] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:14.323572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32106af7-c083-4fb2-9775-f927729da2b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:14.324217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32106af7-c083-4fb2-9775-f927729da2b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:14.324503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32106af7-c083-4fb2-9775-f927729da2b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:14.324786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32106af7-c083-4fb2-9775-f927729da2b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:14.346066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32106af7-c083-4fb2-9775-f927729da2b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:14.350220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32106af7-c083-4fb2-9775-f927729da2b8 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:14.368659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 865/1724] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:14.665652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:15.163792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:15.165333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:15.211333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407be17-8be4-493d-8526-bedcb4296b20 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:15.229608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:15.234356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:15.234356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:15.234356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] PATH_INFO: `/v3/projects/07de55522b6549b6be07de3f5d64d03f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:15.234356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07de55522b6549b6be07de3f5d64d03f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:15.234356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:15.384714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c8a2e-672d-46a3-a402-f00eeab6a4da None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:15.388215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c8a2e-672d-46a3-a402-f00eeab6a4da None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:15.390261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c8a2e-672d-46a3-a402-f00eeab6a4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:15.480477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d10c8a2e-672d-46a3-a402-f00eeab6a4da None None] MFA Rules not processed for user `ce316aa5c0b54ac99ade6cc4163ee209`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:15.483671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d10c8a2e-672d-46a3-a402-f00eeab6a4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:15.487324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07de55522b6549b6be07de3f5d64d03f', 'request_id': 'req-d8e80091-4937-47b4-9036-85e0f67ecb27'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:15.515267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07de55522b6549b6be07de3f5d64d03f', 'request_id': 'req-d8e80091-4937-47b4-9036-85e0f67ecb27'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:15.536824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07de55522b6549b6be07de3f5d64d03f', 'request_id': 'req-d8e80091-4937-47b4-9036-85e0f67ecb27'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:15.652663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] The token cache is being invalidate because project 07de55522b6549b6be07de3f5d64d03f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:15.652995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:15.654484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:15.660324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 860/1725] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:15 2022] DELETE /identity/v3/projects/07de55522b6549b6be07de3f5d64d03f => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:15.664903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:15.666356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:15.732164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 866/1726] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:15.733362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c8a2e-672d-46a3-a402-f00eeab6a4da None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:15.739950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d10c8a2e-672d-46a3-a402-f00eeab6a4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:15.884292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e80091-4937-47b4-9036-85e0f67ecb27 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:15.899875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:15.920806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:15.931740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:15.931740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:15.931740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] PATH_INFO: `/v3/users/ff892413e4b94c9a849869ca4fd9d976` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:15.931740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff892413e4b94c9a849869ca4fd9d976)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:15.931740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:15.947293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c8a2e-672d-46a3-a402-f00eeab6a4da None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:15.958032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:15.958573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:15.958850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:15.959144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] PATH_INFO: `/v3/users/e53e9053a81f4e8884df1e50cd778305` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:15.963887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e53e9053a81f4e8884df1e50cd778305)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:15.965864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:16.202208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff892413e4b94c9a849869ca4fd9d976'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.203097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff892413e4b94c9a849869ca4fd9d976'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.215447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff892413e4b94c9a849869ca4fd9d976'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.216520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff892413e4b94c9a849869ca4fd9d976'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.223104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e53e9053a81f4e8884df1e50cd778305'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.231127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e53e9053a81f4e8884df1e50cd778305'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.231998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e53e9053a81f4e8884df1e50cd778305'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.232799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e53e9053a81f4e8884df1e50cd778305'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.245812 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e53e9053a81f4e8884df1e50cd778305'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.300600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:16.366807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b4cde95c-db67-43a9-87d1-4e13485d76ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff892413e4b94c9a849869ca4fd9d976'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:16.386758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 861/1727] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:15 2022] DELETE /identity/v3/users/ff892413e4b94c9a849869ca4fd9d976 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:16.505613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 867/1728] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:15 2022] DELETE /identity/v3/users/e53e9053a81f4e8884df1e50cd778305 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:16.511965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dac13-8e9c-453e-aa0e-bf1f51ab3c7c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:16.512479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dac13-8e9c-453e-aa0e-bf1f51ab3c7c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:16.512754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dac13-8e9c-453e-aa0e-bf1f51ab3c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:16.546465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4d8dac13-8e9c-453e-aa0e-bf1f51ab3c7c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:16.548050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4d8dac13-8e9c-453e-aa0e-bf1f51ab3c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:16.635054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 862/1729] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:16.688062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:16.724475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:16.727565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:16.807163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46d8db-3534-42f2-b84c-2bad61491c68 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:16.811521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160fd24-3448-46f2-8889-8c1120f38d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:16.812302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160fd24-3448-46f2-8889-8c1120f38d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:16.812890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160fd24-3448-46f2-8889-8c1120f38d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:16.813426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160fd24-3448-46f2-8889-8c1120f38d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:16.826686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160fd24-3448-46f2-8889-8c1120f38d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:16.828470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160fd24-3448-46f2-8889-8c1120f38d19 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:16.839325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 868/1730] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:16.991889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dac13-8e9c-453e-aa0e-bf1f51ab3c7c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:16.993380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4d8dac13-8e9c-453e-aa0e-bf1f51ab3c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:17.012474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dac13-8e9c-453e-aa0e-bf1f51ab3c7c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:17.018833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:17.019511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.019783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.020042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] PATH_INFO: `/v3/projects/6dd180c7801143f39fa02c169b07ae21` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:17.021834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd180c7801143f39fa02c169b07ae21)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:17.024033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:17.165364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd180c7801143f39fa02c169b07ae21', 'request_id': 'req-e207f827-607c-4577-af5c-833d247fce47'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.172711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd180c7801143f39fa02c169b07ae21', 'request_id': 'req-e207f827-607c-4577-af5c-833d247fce47'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.179561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd180c7801143f39fa02c169b07ae21', 'request_id': 'req-e207f827-607c-4577-af5c-833d247fce47'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.298609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] The token cache is being invalidate because project 6dd180c7801143f39fa02c169b07ae21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:17.298903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.305164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e207f827-607c-4577-af5c-833d247fce47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.310334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 863/1731] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:16 2022] DELETE /identity/v3/projects/6dd180c7801143f39fa02c169b07ae21 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:17.316186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae85aa-3328-4e76-a4b9-8d232f9af4e2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.316820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae85aa-3328-4e76-a4b9-8d232f9af4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.317247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae85aa-3328-4e76-a4b9-8d232f9af4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:17.346042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bfae85aa-3328-4e76-a4b9-8d232f9af4e2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:17.347284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bfae85aa-3328-4e76-a4b9-8d232f9af4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:17.423113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 869/1732] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:17.431102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8aa3-6bdf-4fc9-a986-0caa61a169a3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.432198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8aa3-6bdf-4fc9-a986-0caa61a169a3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.432805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8aa3-6bdf-4fc9-a986-0caa61a169a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:17.457149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-42fa8aa3-6bdf-4fc9-a986-0caa61a169a3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:17.458432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-42fa8aa3-6bdf-4fc9-a986-0caa61a169a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:17.467760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 864/1733] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:17.471875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae85aa-3328-4e76-a4b9-8d232f9af4e2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:17.482442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae85aa-3328-4e76-a4b9-8d232f9af4e2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:17.488536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:17.489805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.490274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.490762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:17.491711 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:17.492624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:17.494881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:17.508100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 870/1734] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:17:17.513450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa8aa3-6bdf-4fc9-a986-0caa61a169a3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:17.527613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa8aa3-6bdf-4fc9-a986-0caa61a169a3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:17.532721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a83c3b-bc86-42bf-95fa-ccbad6d1d88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:17.533330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a83c3b-bc86-42bf-95fa-ccbad6d1d88a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.533728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a83c3b-bc86-42bf-95fa-ccbad6d1d88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.534172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a83c3b-bc86-42bf-95fa-ccbad6d1d88a admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:17.534805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a83c3b-bc86-42bf-95fa-ccbad6d1d88a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:17.542184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a83c3b-bc86-42bf-95fa-ccbad6d1d88a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:17.604473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:17.606589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:17.628540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a222a-8938-43c2-8177-ff224fd09c44 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:17.643061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:17.644085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.644820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.645010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] PATH_INFO: `/v3/projects/6e70945a174d4af4abedd30925eb4130` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:17.646987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e70945a174d4af4abedd30925eb4130)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:17.651789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:17.680378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 865/1735] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:17 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:17.684116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a83c3b-bc86-42bf-95fa-ccbad6d1d88a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:17.799526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a83c3b-bc86-42bf-95fa-ccbad6d1d88a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:17.811111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e70945a174d4af4abedd30925eb4130', 'request_id': 'req-0b3e6562-0928-4d84-9c81-593ab7aec506'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.818975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:17.889873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:17.890478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.890757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.891055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:17.891868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:17.893886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:17.901769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e70945a174d4af4abedd30925eb4130', 'request_id': 'req-0b3e6562-0928-4d84-9c81-593ab7aec506'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.909103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e70945a174d4af4abedd30925eb4130', 'request_id': 'req-0b3e6562-0928-4d84-9c81-593ab7aec506'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.922804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] The token cache is being invalidate because project 6e70945a174d4af4abedd30925eb4130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:17.923248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.923986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b3e6562-0928-4d84-9c81-593ab7aec506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:17.925692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 871/1736] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:17 2022] DELETE /identity/v3/projects/6e70945a174d4af4abedd30925eb4130 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:17.932674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb3d43-cb02-4eaf-bcc3-deebf4ee9ac8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.933121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb3d43-cb02-4eaf-bcc3-deebf4ee9ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.933551 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb3d43-cb02-4eaf-bcc3-deebf4ee9ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:17.958032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1acb3d43-cb02-4eaf-bcc3-deebf4ee9ac8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:17.959413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1acb3d43-cb02-4eaf-bcc3-deebf4ee9ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:17.975010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 872/1737] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:17.976442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a23bb-5e0b-4cbe-af86-36a7a8190a8c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:17.977038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a23bb-5e0b-4cbe-af86-36a7a8190a8c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:17.977492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a23bb-5e0b-4cbe-af86-36a7a8190a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.000905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-507a23bb-5e0b-4cbe-af86-36a7a8190a8c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:18.002220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-507a23bb-5e0b-4cbe-af86-36a7a8190a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:18.006181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 866/1738] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:17 2022] POST /identity/v3/users => generated 288 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:18.008874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.010631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:18.012519 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 873/1739] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:18.016516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a23bb-5e0b-4cbe-af86-36a7a8190a8c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.023649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824b700-c54c-4cdb-bdd5-dce8af31ccb9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.026549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a23bb-5e0b-4cbe-af86-36a7a8190a8c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.029192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199ed21-16be-4dfa-aebe-152548ce5ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.029799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199ed21-16be-4dfa-aebe-152548ce5ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.030081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199ed21-16be-4dfa-aebe-152548ce5ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.030350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199ed21-16be-4dfa-aebe-152548ce5ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.031020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199ed21-16be-4dfa-aebe-152548ce5ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.032388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.032620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199ed21-16be-4dfa-aebe-152548ce5ef8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.033163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.033559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.033983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.034772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:18.035145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.036896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.045478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 867/1740] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:18.049015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 874/1741] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:17:18.051480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199ed21-16be-4dfa-aebe-152548ce5ef8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.053677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.062008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199ed21-16be-4dfa-aebe-152548ce5ef8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.063371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22576a15-39ea-49a0-b599-b97a3dd599ce admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.067000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65876d9-8051-4c60-baf0-ff190cb0d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.067802 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65876d9-8051-4c60-baf0-ff190cb0d363 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.068117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65876d9-8051-4c60-baf0-ff190cb0d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.068358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b7be7-2e67-4fa2-a7d5-49c63eedbd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.068475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65876d9-8051-4c60-baf0-ff190cb0d363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.068946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7be7-2e67-4fa2-a7d5-49c63eedbd42 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.069108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65876d9-8051-4c60-baf0-ff190cb0d363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.069391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7be7-2e67-4fa2-a7d5-49c63eedbd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.069662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7be7-2e67-4fa2-a7d5-49c63eedbd42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.070285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b7be7-2e67-4fa2-a7d5-49c63eedbd42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.071078 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65876d9-8051-4c60-baf0-ff190cb0d363 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.072216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b7be7-2e67-4fa2-a7d5-49c63eedbd42 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.086431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 868/1742] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:17:18.089323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65876d9-8051-4c60-baf0-ff190cb0d363 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.100208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65876d9-8051-4c60-baf0-ff190cb0d363 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.104945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f475157-7a5c-4406-9fe2-4f6aec6f7499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.105580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475157-7a5c-4406-9fe2-4f6aec6f7499 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.105855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475157-7a5c-4406-9fe2-4f6aec6f7499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.106123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475157-7a5c-4406-9fe2-4f6aec6f7499 admin admin] PATH_INFO: `/v3/projects/dd4db37a4b194e129570876b1e277dfd/users/13c1b99d6b5b4847a1c3c1aff74facc0/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.120742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f475157-7a5c-4406-9fe2-4f6aec6f7499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4db37a4b194e129570876b1e277dfd, user_id=13c1b99d6b5b4847a1c3c1aff74facc0, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.123473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f475157-7a5c-4406-9fe2-4f6aec6f7499 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.175657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 875/1743] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:18 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:18.179840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b7be7-2e67-4fa2-a7d5-49c63eedbd42 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.204372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 869/1744] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:18 2022] PUT /identity/v3/projects/dd4db37a4b194e129570876b1e277dfd/users/13c1b99d6b5b4847a1c3c1aff74facc0/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:18.214547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81850b0a-b884-47b8-9e56-7b14a18a5fd0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.214837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81850b0a-b884-47b8-9e56-7b14a18a5fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.215162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81850b0a-b884-47b8-9e56-7b14a18a5fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.249533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-81850b0a-b884-47b8-9e56-7b14a18a5fd0 None None] MFA Rules not processed for user `13c1b99d6b5b4847a1c3c1aff74facc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:18.250921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-81850b0a-b884-47b8-9e56-7b14a18a5fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:18.255945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b7be7-2e67-4fa2-a7d5-49c63eedbd42 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.315534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef4e6f-4512-429f-bf0f-9e7f142f3fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.316725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4e6f-4512-429f-bf0f-9e7f142f3fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.316725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4e6f-4512-429f-bf0f-9e7f142f3fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.316725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef4e6f-4512-429f-bf0f-9e7f142f3fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.317506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef4e6f-4512-429f-bf0f-9e7f142f3fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.319572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef4e6f-4512-429f-bf0f-9e7f142f3fc0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.319878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 870/1745] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:18.331406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81850b0a-b884-47b8-9e56-7b14a18a5fd0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.333628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-81850b0a-b884-47b8-9e56-7b14a18a5fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:18.389201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81850b0a-b884-47b8-9e56-7b14a18a5fd0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.392768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a746f-9ebd-4fe3-87cd-9620312933e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.393409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a746f-9ebd-4fe3-87cd-9620312933e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.393686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a746f-9ebd-4fe3-87cd-9620312933e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.393958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a746f-9ebd-4fe3-87cd-9620312933e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.402219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a746f-9ebd-4fe3-87cd-9620312933e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.403745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a746f-9ebd-4fe3-87cd-9620312933e0 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.414082 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 871/1746] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:17:18.448453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 876/1747] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:18 2022] POST /identity/v3/users => generated 303 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:18.452429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a746f-9ebd-4fe3-87cd-9620312933e0 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.463147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a746f-9ebd-4fe3-87cd-9620312933e0 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.467886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768f12-52aa-4bfa-8827-9254d3871a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.468533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768f12-52aa-4bfa-8827-9254d3871a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.468811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768f12-52aa-4bfa-8827-9254d3871a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.469085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38768f12-52aa-4bfa-8827-9254d3871a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.469759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38768f12-52aa-4bfa-8827-9254d3871a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.471916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38768f12-52aa-4bfa-8827-9254d3871a26 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.482510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 872/1748] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:18.493783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef4e6f-4512-429f-bf0f-9e7f142f3fc0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.507192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef4e6f-4512-429f-bf0f-9e7f142f3fc0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.512142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7850cb3-5778-4854-9fe1-fa6eb440a80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.512813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850cb3-5778-4854-9fe1-fa6eb440a80c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.513213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850cb3-5778-4854-9fe1-fa6eb440a80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.513505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850cb3-5778-4854-9fe1-fa6eb440a80c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.514898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7850cb3-5778-4854-9fe1-fa6eb440a80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.516021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7850cb3-5778-4854-9fe1-fa6eb440a80c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.526251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 877/1749] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:18.530482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768f12-52aa-4bfa-8827-9254d3871a26 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.540971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38768f12-52aa-4bfa-8827-9254d3871a26 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.545624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.546269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.546555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.546809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] PATH_INFO: `/v3/projects/c8bbe7826f834efaaba7d8823b897209/users/aa9637c50743421b8e17ffc6f22c55ba/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.562362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bbe7826f834efaaba7d8823b897209, user_id=aa9637c50743421b8e17ffc6f22c55ba, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.567652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.716431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 873/1750] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:18 2022] PUT /identity/v3/projects/c8bbe7826f834efaaba7d8823b897209/users/aa9637c50743421b8e17ffc6f22c55ba/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:18.723517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69398013-c316-4c63-8056-b539f5b5808a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.727197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69398013-c316-4c63-8056-b539f5b5808a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.727486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69398013-c316-4c63-8056-b539f5b5808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.746066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-69398013-c316-4c63-8056-b539f5b5808a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:18.747283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-69398013-c316-4c63-8056-b539f5b5808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:18.811445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 878/1751] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:18.821871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:18.824823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:18.881295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:18.886221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ddc7-6cbc-45a3-8208-605ee1591284 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:18.889991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:18.890687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:18.890874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:18.891173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:18.900914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:18.902335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:18.912017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 874/1752] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:19.180412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31b435-cf47-4046-bc68-96cee68cdc3d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:19.180700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31b435-cf47-4046-bc68-96cee68cdc3d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:19.180971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31b435-cf47-4046-bc68-96cee68cdc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:19.201270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5c31b435-cf47-4046-bc68-96cee68cdc3d None None] MFA Rules not processed for user `d99632164b3b4a1f95736a65da217db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:19.202493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5c31b435-cf47-4046-bc68-96cee68cdc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:19.209034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 879/1753] 10.222.0.59 () {58 vars in 1015 bytes} [Tue Aug 23 08:17:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:19.229791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:19.231309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:19.244409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27aac9-de49-47f1-bfa5-296c4c2d4693 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:19.250094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688e450-a480-40a6-b8e0-92c7f1350aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:19.250872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e450-a480-40a6-b8e0-92c7f1350aba service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:19.251403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e450-a480-40a6-b8e0-92c7f1350aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:19.251774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e450-a480-40a6-b8e0-92c7f1350aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:19.260193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688e450-a480-40a6-b8e0-92c7f1350aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:19.265540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688e450-a480-40a6-b8e0-92c7f1350aba service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:19.279205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 875/1754] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:20.236585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fff1b4-dd43-4de9-a9eb-0f4512b9007a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:20.238010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fff1b4-dd43-4de9-a9eb-0f4512b9007a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:20.238539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fff1b4-dd43-4de9-a9eb-0f4512b9007a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:20.318733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f1fff1b4-dd43-4de9-a9eb-0f4512b9007a None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:20.320062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f1fff1b4-dd43-4de9-a9eb-0f4512b9007a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:20.408104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 880/1755] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:17:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:20.413813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:20.587166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b8ed-37d1-4e15-b9e8-10ff641a4282 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:20.587672 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b8ed-37d1-4e15-b9e8-10ff641a4282 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:20.588094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b8ed-37d1-4e15-b9e8-10ff641a4282 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:20.593854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 876/1756] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:17:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:17:20.618229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2202a-472a-4e15-97a1-e78393e70692 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:20.618229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2202a-472a-4e15-97a1-e78393e70692 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:20.618440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2202a-472a-4e15-97a1-e78393e70692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:20.640318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1fd2202a-472a-4e15-97a1-e78393e70692 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:20.643957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1fd2202a-472a-4e15-97a1-e78393e70692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:20.752071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 881/1757] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:17:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:20.771745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51b8ed-37d1-4e15-b9e8-10ff641a4282 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:20.773634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7e51b8ed-37d1-4e15-b9e8-10ff641a4282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:20.791003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2202a-472a-4e15-97a1-e78393e70692 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:20.794547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1fd2202a-472a-4e15-97a1-e78393e70692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:20.817412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51b8ed-37d1-4e15-b9e8-10ff641a4282 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:20.840425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd1de-07ef-424e-b518-5f702a83d678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:20.841047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd1de-07ef-424e-b518-5f702a83d678 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:20.841555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd1de-07ef-424e-b518-5f702a83d678 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:20.842165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd1de-07ef-424e-b518-5f702a83d678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:20.861932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd1de-07ef-424e-b518-5f702a83d678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:20.867248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd1de-07ef-424e-b518-5f702a83d678 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:20.895490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 877/1758] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:21.040901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2202a-472a-4e15-97a1-e78393e70692 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:21.068970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a335ea-2af7-4e6a-aca9-d3c527fe83c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:21.069899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a335ea-2af7-4e6a-aca9-d3c527fe83c4 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:21.071413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a335ea-2af7-4e6a-aca9-d3c527fe83c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:21.072333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a335ea-2af7-4e6a-aca9-d3c527fe83c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:21.091142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a335ea-2af7-4e6a-aca9-d3c527fe83c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:21.093124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a335ea-2af7-4e6a-aca9-d3c527fe83c4 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:21.124115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 882/1759] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:17:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 08:17:21.328522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65d44a-b1f0-4a13-a2bf-6d20d9b98f42 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:21.329649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65d44a-b1f0-4a13-a2bf-6d20d9b98f42 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:21.330309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65d44a-b1f0-4a13-a2bf-6d20d9b98f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:21.419950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-db65d44a-b1f0-4a13-a2bf-6d20d9b98f42 None None] MFA Rules not processed for user `aa9637c50743421b8e17ffc6f22c55ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:21.421241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-db65d44a-b1f0-4a13-a2bf-6d20d9b98f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:21.582334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 878/1760] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:21 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:21.583724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a335ea-2af7-4e6a-aca9-d3c527fe83c4 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:21.600501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a335ea-2af7-4e6a-aca9-d3c527fe83c4 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:21.608264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4e109-1278-4ef1-b91b-a95dec040b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:21.609048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e109-1278-4ef1-b91b-a95dec040b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:21.609434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e109-1278-4ef1-b91b-a95dec040b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:21.609720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4e109-1278-4ef1-b91b-a95dec040b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:21.610378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4e109-1278-4ef1-b91b-a95dec040b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:21.614871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4e109-1278-4ef1-b91b-a95dec040b95 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:21.804154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 883/1761] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:21 2022] POST /identity/v3/projects => generated 403 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:21.805687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65d44a-b1f0-4a13-a2bf-6d20d9b98f42 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:21.826027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65d44a-b1f0-4a13-a2bf-6d20d9b98f42 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:22.052573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25351a-dc29-42d5-b020-5195b7cbeb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:22.052573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25351a-dc29-42d5-b020-5195b7cbeb58 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:22.052573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25351a-dc29-42d5-b020-5195b7cbeb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:22.052573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25351a-dc29-42d5-b020-5195b7cbeb58 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:22.053003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25351a-dc29-42d5-b020-5195b7cbeb58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:22.058585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25351a-dc29-42d5-b020-5195b7cbeb58 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:22.230827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 879/1762] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:21 2022] POST /identity/v3/users => generated 308 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:22.232343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4e109-1278-4ef1-b91b-a95dec040b95 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:22.254436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4e109-1278-4ef1-b91b-a95dec040b95 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:22.255603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:22.269243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260a7d7-8635-4b26-b62e-00f1c2095f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:22.269896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260a7d7-8635-4b26-b62e-00f1c2095f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:22.270356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260a7d7-8635-4b26-b62e-00f1c2095f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:22.270662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260a7d7-8635-4b26-b62e-00f1c2095f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:22.271349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260a7d7-8635-4b26-b62e-00f1c2095f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:22.277473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260a7d7-8635-4b26-b62e-00f1c2095f53 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:22.302449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 884/1763] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:22.302638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25351a-dc29-42d5-b020-5195b7cbeb58 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:22.316418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25351a-dc29-42d5-b020-5195b7cbeb58 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:22.321623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f60c03-8d9e-4cde-b9dd-9d91f0a2ef93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:22.322521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f60c03-8d9e-4cde-b9dd-9d91f0a2ef93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:22.322979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f60c03-8d9e-4cde-b9dd-9d91f0a2ef93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:22.323430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f60c03-8d9e-4cde-b9dd-9d91f0a2ef93 admin admin] PATH_INFO: `/v3/projects/739b4e7307bb417bb2cb53d14d4db8bc/users/c7086fb8c1e54e5983ad54fd29136ba6/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:22.353074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f60c03-8d9e-4cde-b9dd-9d91f0a2ef93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=739b4e7307bb417bb2cb53d14d4db8bc, user_id=c7086fb8c1e54e5983ad54fd29136ba6, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:22.357984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f60c03-8d9e-4cde-b9dd-9d91f0a2ef93 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:22.482847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 880/1764] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:22 2022] PUT /identity/v3/projects/739b4e7307bb417bb2cb53d14d4db8bc/users/c7086fb8c1e54e5983ad54fd29136ba6/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:22.486672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260a7d7-8635-4b26-b62e-00f1c2095f53 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:22.501255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260a7d7-8635-4b26-b62e-00f1c2095f53 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:22.634822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1065bc-8718-4fd7-8170-871ae20be1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:22.701589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1065bc-8718-4fd7-8170-871ae20be1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:22.701589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1065bc-8718-4fd7-8170-871ae20be1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:22.701589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1065bc-8718-4fd7-8170-871ae20be1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:22.701589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1065bc-8718-4fd7-8170-871ae20be1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:22.701589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1065bc-8718-4fd7-8170-871ae20be1e0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:22.701589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 885/1765] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:22.712742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f60c03-8d9e-4cde-b9dd-9d91f0a2ef93 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:22.728963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f60c03-8d9e-4cde-b9dd-9d91f0a2ef93 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:22.745290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1d3e5-1c85-4e32-b204-9073bdd5f0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:22.750150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1d3e5-1c85-4e32-b204-9073bdd5f0f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:22.750150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1d3e5-1c85-4e32-b204-9073bdd5f0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:22.750150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1d3e5-1c85-4e32-b204-9073bdd5f0f0 admin admin] PATH_INFO: `/v3/projects/739b4e7307bb417bb2cb53d14d4db8bc/users/c7086fb8c1e54e5983ad54fd29136ba6/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:22.750150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1d3e5-1c85-4e32-b204-9073bdd5f0f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=739b4e7307bb417bb2cb53d14d4db8bc, user_id=c7086fb8c1e54e5983ad54fd29136ba6, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:22.752752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1d3e5-1c85-4e32-b204-9073bdd5f0f0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:22.774494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 881/1766] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:22 2022] PUT /identity/v3/projects/739b4e7307bb417bb2cb53d14d4db8bc/users/c7086fb8c1e54e5983ad54fd29136ba6/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:23.727167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:25.220076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:25.858032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688b94-c6cb-4e8d-926d-bb97cb2a5f08 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:25.858324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688b94-c6cb-4e8d-926d-bb97cb2a5f08 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:25.858607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688b94-c6cb-4e8d-926d-bb97cb2a5f08 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:25.860011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 886/1767] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:17:25.869972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153f92c-dedd-42a1-9c7f-75c94b050d60 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:25.870533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153f92c-dedd-42a1-9c7f-75c94b050d60 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:25.870533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153f92c-dedd-42a1-9c7f-75c94b050d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:25.895192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8153f92c-dedd-42a1-9c7f-75c94b050d60 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:25.895560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8153f92c-dedd-42a1-9c7f-75c94b050d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:25.975959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 882/1768] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:17:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:17:25.991260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688b94-c6cb-4e8d-926d-bb97cb2a5f08 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:26.009885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688b94-c6cb-4e8d-926d-bb97cb2a5f08 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:26.016226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65f298-839d-42a8-aaa3-f12e14425dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:26.017016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65f298-839d-42a8-aaa3-f12e14425dcd service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:26.017464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65f298-839d-42a8-aaa3-f12e14425dcd service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:26.017891 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65f298-839d-42a8-aaa3-f12e14425dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:26.028453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65f298-839d-42a8-aaa3-f12e14425dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:26.030524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65f298-839d-42a8-aaa3-f12e14425dcd service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:26.042924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 887/1769] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:26.101445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076323e2-af2e-4848-a3fd-aff73f94f31f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:26.101734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076323e2-af2e-4848-a3fd-aff73f94f31f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:26.102125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076323e2-af2e-4848-a3fd-aff73f94f31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:26.135013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-076323e2-af2e-4848-a3fd-aff73f94f31f None None] MFA Rules not processed for user `c7086fb8c1e54e5983ad54fd29136ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:26.136894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-076323e2-af2e-4848-a3fd-aff73f94f31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:26.231348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 883/1770] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:26 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:17:26.532004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:27.056010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318a504-3c41-4d35-b36d-622156cbeb60 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:27.056555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318a504-3c41-4d35-b36d-622156cbeb60 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:27.056555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318a504-3c41-4d35-b36d-622156cbeb60 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:27.057940 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 888/1771] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:17:27.078460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e664a-8117-42f8-82ae-2b0eb75c167f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:27.078979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e664a-8117-42f8-82ae-2b0eb75c167f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:27.079136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e664a-8117-42f8-82ae-2b0eb75c167f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:27.100179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ed1e664a-8117-42f8-82ae-2b0eb75c167f None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:27.103839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ed1e664a-8117-42f8-82ae-2b0eb75c167f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:27.166545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 884/1772] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:17:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:27.178254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318a504-3c41-4d35-b36d-622156cbeb60 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:27.180050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5318a504-3c41-4d35-b36d-622156cbeb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:27.196893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318a504-3c41-4d35-b36d-622156cbeb60 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:27.202795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593e6c6-4dbb-4cbe-a9d0-91aa13b6b091 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:27.203499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593e6c6-4dbb-4cbe-a9d0-91aa13b6b091 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:27.203848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593e6c6-4dbb-4cbe-a9d0-91aa13b6b091 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:27.204243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593e6c6-4dbb-4cbe-a9d0-91aa13b6b091 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:27.213510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593e6c6-4dbb-4cbe-a9d0-91aa13b6b091 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:27.216244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593e6c6-4dbb-4cbe-a9d0-91aa13b6b091 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:27.238520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 889/1773] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:27.328451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:27.565994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:28.022135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e664a-8117-42f8-82ae-2b0eb75c167f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:28.102344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e664a-8117-42f8-82ae-2b0eb75c167f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:28.118269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:28.118269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:28.118269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:28.118269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] PATH_INFO: `/v3/users/aa9637c50743421b8e17ffc6f22c55ba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:28.118615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9637c50743421b8e17ffc6f22c55ba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:28.118615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:28.420595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9637c50743421b8e17ffc6f22c55ba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:28.421500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9637c50743421b8e17ffc6f22c55ba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:28.428499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9637c50743421b8e17ffc6f22c55ba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:28.430703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9637c50743421b8e17ffc6f22c55ba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:28.505473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9637c50743421b8e17ffc6f22c55ba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:28.523723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 885/1774] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:28 2022] DELETE /identity/v3/users/aa9637c50743421b8e17ffc6f22c55ba => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:28.628999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:28.776883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593e6c6-4dbb-4cbe-a9d0-91aa13b6b091 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:28.791707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593e6c6-4dbb-4cbe-a9d0-91aa13b6b091 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:28.849385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:28.850007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:28.850451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:28.850729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] PATH_INFO: `/v3/projects/c8bbe7826f834efaaba7d8823b897209` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:28.851907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8bbe7826f834efaaba7d8823b897209)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:28.853982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:28.917461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:28.976184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8bbe7826f834efaaba7d8823b897209', 'request_id': 'req-f79e2f7f-8fef-4cbf-930e-7df57584323a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.036751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8bbe7826f834efaaba7d8823b897209', 'request_id': 'req-f79e2f7f-8fef-4cbf-930e-7df57584323a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.046054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8bbe7826f834efaaba7d8823b897209', 'request_id': 'req-f79e2f7f-8fef-4cbf-930e-7df57584323a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.059725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] The token cache is being invalidate because project c8bbe7826f834efaaba7d8823b897209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:29.060347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.061385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.064092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 890/1775] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:28 2022] DELETE /identity/v3/projects/c8bbe7826f834efaaba7d8823b897209 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:29.067772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:29.070298 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:29.091774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450bab8-b9dd-4f81-9ed0-c417f3141f6d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:29.184165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:29.185503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:29.185958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:29.186372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] PATH_INFO: `/v3/users/c7086fb8c1e54e5983ad54fd29136ba6` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:29.191603 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7086fb8c1e54e5983ad54fd29136ba6)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:29.193781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:29.452280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7086fb8c1e54e5983ad54fd29136ba6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.453228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7086fb8c1e54e5983ad54fd29136ba6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.465017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7086fb8c1e54e5983ad54fd29136ba6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.466219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7086fb8c1e54e5983ad54fd29136ba6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.568396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4023f71e-a882-47e6-9488-9279ed82e49e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7086fb8c1e54e5983ad54fd29136ba6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:29.593844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 886/1776] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:29 2022] DELETE /identity/v3/users/c7086fb8c1e54e5983ad54fd29136ba6 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:29.935316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:29.936820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:29.953178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e2f7f-8fef-4cbf-930e-7df57584323a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:30.014130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:30.014756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:30.015094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:30.015343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] PATH_INFO: `/v3/projects/739b4e7307bb417bb2cb53d14d4db8bc` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:30.016488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] RBAC: Authorizing `identity:delete_project(project_id=739b4e7307bb417bb2cb53d14d4db8bc)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:30.018399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:30.042642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:30.164408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '739b4e7307bb417bb2cb53d14d4db8bc', 'request_id': 'req-d4ae93ac-6569-440c-9671-21976b74510a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:30.305861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '739b4e7307bb417bb2cb53d14d4db8bc', 'request_id': 'req-d4ae93ac-6569-440c-9671-21976b74510a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:30.313127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '739b4e7307bb417bb2cb53d14d4db8bc', 'request_id': 'req-d4ae93ac-6569-440c-9671-21976b74510a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:30.326548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] The token cache is being invalidate because project 739b4e7307bb417bb2cb53d14d4db8bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:30.326986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:30.327678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d4ae93ac-6569-440c-9671-21976b74510a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:30.337322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 891/1777] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:29 2022] DELETE /identity/v3/projects/739b4e7307bb417bb2cb53d14d4db8bc => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:30.341838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c5ca1-da1c-4b0b-b3ac-027ba5f2adb0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:30.342683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c5ca1-da1c-4b0b-b3ac-027ba5f2adb0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:30.342990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c5ca1-da1c-4b0b-b3ac-027ba5f2adb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:30.381793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-936c5ca1-da1c-4b0b-b3ac-027ba5f2adb0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:30.383307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-936c5ca1-da1c-4b0b-b3ac-027ba5f2adb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:30.489425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 887/1778] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:30.496834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191cec0-e863-4b99-827d-fce33ecb3894 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:30.497119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191cec0-e863-4b99-827d-fce33ecb3894 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:30.497393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191cec0-e863-4b99-827d-fce33ecb3894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:30.519945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5191cec0-e863-4b99-827d-fce33ecb3894 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:30.521219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5191cec0-e863-4b99-827d-fce33ecb3894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:30.529651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 892/1779] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:30.534734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c5ca1-da1c-4b0b-b3ac-027ba5f2adb0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:30.546585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c5ca1-da1c-4b0b-b3ac-027ba5f2adb0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:30.550366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:30.551020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:30.551307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:30.551581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:30.552268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:30.552575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:30.554254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:30.571001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 888/1780] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:17:30.575705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191cec0-e863-4b99-827d-fce33ecb3894 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:30.589882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191cec0-e863-4b99-827d-fce33ecb3894 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:30.601113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365c01f-579a-4234-a340-af66267c48a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:30.604954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365c01f-579a-4234-a340-af66267c48a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:30.605485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365c01f-579a-4234-a340-af66267c48a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:30.605608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365c01f-579a-4234-a340-af66267c48a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:30.606296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365c01f-579a-4234-a340-af66267c48a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:30.608434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365c01f-579a-4234-a340-af66267c48a4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:30.745365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 893/1781] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:30 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:30.746756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:30.758975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eac199-6016-4f42-9ce3-88b575b1fc20 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:30.763472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:30.822380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad87b5f-cae0-4150-a855-462c3818e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:30.823186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad87b5f-cae0-4150-a855-462c3818e01c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:30.823878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad87b5f-cae0-4150-a855-462c3818e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:30.823878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad87b5f-cae0-4150-a855-462c3818e01c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:30.824463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad87b5f-cae0-4150-a855-462c3818e01c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:30.826725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad87b5f-cae0-4150-a855-462c3818e01c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:30.939174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 889/1782] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:30 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:30.946676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365c01f-579a-4234-a340-af66267c48a4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:30.956100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365c01f-579a-4234-a340-af66267c48a4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:30.961277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d533da-37f1-4ae9-ad21-659dd97e5e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:30.961880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d533da-37f1-4ae9-ad21-659dd97e5e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:30.962151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d533da-37f1-4ae9-ad21-659dd97e5e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:30.962578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d533da-37f1-4ae9-ad21-659dd97e5e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:30.963433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d533da-37f1-4ae9-ad21-659dd97e5e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:30.965033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d533da-37f1-4ae9-ad21-659dd97e5e4c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:30.992938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad87b5f-cae0-4150-a855-462c3818e01c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:30.993982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 894/1783] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:31.003562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad87b5f-cae0-4150-a855-462c3818e01c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:31.011921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158631e-7e16-4ff0-9bdb-bc48ce07eb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:31.012554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158631e-7e16-4ff0-9bdb-bc48ce07eb9e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:31.012833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158631e-7e16-4ff0-9bdb-bc48ce07eb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:31.013106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158631e-7e16-4ff0-9bdb-bc48ce07eb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:31.013760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158631e-7e16-4ff0-9bdb-bc48ce07eb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:31.015650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158631e-7e16-4ff0-9bdb-bc48ce07eb9e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:31.031231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 890/1784] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:31.035448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d533da-37f1-4ae9-ad21-659dd97e5e4c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:31.045690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d533da-37f1-4ae9-ad21-659dd97e5e4c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:31.050669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601f8cc-a614-4f96-a8d1-20e7163c2b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:31.051344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f8cc-a614-4f96-a8d1-20e7163c2b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:31.051621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f8cc-a614-4f96-a8d1-20e7163c2b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:31.051890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f8cc-a614-4f96-a8d1-20e7163c2b41 admin admin] PATH_INFO: `/v3/projects/df2262b66c8f4020bc0d05cfb320bea1/users/ff073f4cdfca4fde8c76c71b616909c7/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:31.068097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601f8cc-a614-4f96-a8d1-20e7163c2b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2262b66c8f4020bc0d05cfb320bea1, user_id=ff073f4cdfca4fde8c76c71b616909c7, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:31.071491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601f8cc-a614-4f96-a8d1-20e7163c2b41 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:31.209819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 895/1785] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:31 2022] PUT /identity/v3/projects/df2262b66c8f4020bc0d05cfb320bea1/users/ff073f4cdfca4fde8c76c71b616909c7/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:31.221218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a201-3d0d-43ca-ac9d-1ee2f779d908 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:31.221502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a201-3d0d-43ca-ac9d-1ee2f779d908 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:31.221777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a201-3d0d-43ca-ac9d-1ee2f779d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:31.252046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-02d6a201-3d0d-43ca-ac9d-1ee2f779d908 None None] MFA Rules not processed for user `ff073f4cdfca4fde8c76c71b616909c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:31.253322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-02d6a201-3d0d-43ca-ac9d-1ee2f779d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:31.324273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 891/1786] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:31.329553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601f8cc-a614-4f96-a8d1-20e7163c2b41 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:31.444560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601f8cc-a614-4f96-a8d1-20e7163c2b41 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:31.450465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c994339-7dff-48ac-867c-1cf24bef5c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:31.451395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c994339-7dff-48ac-867c-1cf24bef5c8e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:31.451892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c994339-7dff-48ac-867c-1cf24bef5c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:31.452164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c994339-7dff-48ac-867c-1cf24bef5c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:31.452951 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c994339-7dff-48ac-867c-1cf24bef5c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:31.454905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c994339-7dff-48ac-867c-1cf24bef5c8e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:31.613571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 896/1787] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:31 2022] POST /identity/v3/projects => generated 381 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:31.615059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a201-3d0d-43ca-ac9d-1ee2f779d908 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:31.640619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a201-3d0d-43ca-ac9d-1ee2f779d908 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:31.735551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757915e-601d-4fc0-afc5-a7971f54d6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:31.736234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757915e-601d-4fc0-afc5-a7971f54d6bc admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:31.736439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757915e-601d-4fc0-afc5-a7971f54d6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:31.736710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757915e-601d-4fc0-afc5-a7971f54d6bc admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:31.737353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757915e-601d-4fc0-afc5-a7971f54d6bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:31.739356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757915e-601d-4fc0-afc5-a7971f54d6bc admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:31.880219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 892/1788] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:31 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:31.898845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c994339-7dff-48ac-867c-1cf24bef5c8e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:31.919524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c994339-7dff-48ac-867c-1cf24bef5c8e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:31.930163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460eb301-ec46-4120-acb3-3c52fe5fb8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:31.931161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460eb301-ec46-4120-acb3-3c52fe5fb8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:31.931545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460eb301-ec46-4120-acb3-3c52fe5fb8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:31.931925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460eb301-ec46-4120-acb3-3c52fe5fb8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:31.933023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460eb301-ec46-4120-acb3-3c52fe5fb8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:31.934880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460eb301-ec46-4120-acb3-3c52fe5fb8b3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:31.947321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 897/1789] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:31.956490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757915e-601d-4fc0-afc5-a7971f54d6bc admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:31.970161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757915e-601d-4fc0-afc5-a7971f54d6bc admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:31.982373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260a003-f198-463f-9725-5964ce8e8b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:31.983248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260a003-f198-463f-9725-5964ce8e8b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:31.983530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260a003-f198-463f-9725-5964ce8e8b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:31.984199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260a003-f198-463f-9725-5964ce8e8b6b admin admin] PATH_INFO: `/v3/projects/8d7ede3fff2341d0941ae28aa572dcc9/users/030d61b39e7d43529c0a0a1478d64dda/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:32.055880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260a003-f198-463f-9725-5964ce8e8b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7ede3fff2341d0941ae28aa572dcc9, user_id=030d61b39e7d43529c0a0a1478d64dda, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:32.060951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260a003-f198-463f-9725-5964ce8e8b6b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:32.146526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 893/1790] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:31 2022] PUT /identity/v3/projects/8d7ede3fff2341d0941ae28aa572dcc9/users/030d61b39e7d43529c0a0a1478d64dda/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:32.151401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460eb301-ec46-4120-acb3-3c52fe5fb8b3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:32.163630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460eb301-ec46-4120-acb3-3c52fe5fb8b3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:32.284403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fdf48-0023-4105-960d-632603a7db62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:32.285305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fdf48-0023-4105-960d-632603a7db62 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:32.285729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fdf48-0023-4105-960d-632603a7db62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:32.286145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fdf48-0023-4105-960d-632603a7db62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:32.286973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fdf48-0023-4105-960d-632603a7db62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:32.289157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fdf48-0023-4105-960d-632603a7db62 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:32.303127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 898/1791] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:32.307193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260a003-f198-463f-9725-5964ce8e8b6b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:32.321495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260a003-f198-463f-9725-5964ce8e8b6b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:32.342748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:32.346729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:32.346729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:32.346729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] PATH_INFO: `/v3/projects/8d7ede3fff2341d0941ae28aa572dcc9/users/030d61b39e7d43529c0a0a1478d64dda/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:32.348525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7ede3fff2341d0941ae28aa572dcc9, user_id=030d61b39e7d43529c0a0a1478d64dda, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:32.350462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:32.392265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 894/1792] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:32 2022] PUT /identity/v3/projects/8d7ede3fff2341d0941ae28aa572dcc9/users/030d61b39e7d43529c0a0a1478d64dda/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:32.397534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb7529-dd05-47a2-a0bb-bcc08a3a73af None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:32.398069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb7529-dd05-47a2-a0bb-bcc08a3a73af None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:32.398565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb7529-dd05-47a2-a0bb-bcc08a3a73af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:32.468149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-13fb7529-dd05-47a2-a0bb-bcc08a3a73af None None] MFA Rules not processed for user `030d61b39e7d43529c0a0a1478d64dda`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:32.469416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-13fb7529-dd05-47a2-a0bb-bcc08a3a73af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:32.551530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 899/1793] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:32.562087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:32.567732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:32.646963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:32.683609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ab30d-daf1-4f71-ad5c-e6b405a8f5b4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:32.693262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:32.693887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:32.694309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:32.694678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:32.727015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:32.745874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:32.775570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 895/1794] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:33.015771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb7529-dd05-47a2-a0bb-bcc08a3a73af None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:33.025579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb7529-dd05-47a2-a0bb-bcc08a3a73af None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:33.030038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b36d7-b73e-49e8-9008-38b4791ff482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:33.030846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b36d7-b73e-49e8-9008-38b4791ff482 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:33.031174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b36d7-b73e-49e8-9008-38b4791ff482 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:33.031471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b36d7-b73e-49e8-9008-38b4791ff482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:33.091926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b36d7-b73e-49e8-9008-38b4791ff482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:33.093369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b36d7-b73e-49e8-9008-38b4791ff482 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:33.100551 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:33.105701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 900/1795] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:34.132392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:34.320818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:43.903965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:43.905586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:43.971929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3030b8-f5bb-430b-b5d9-06560bc4fe70 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:43.976514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:43.977138 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:43.977411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:43.977678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] PATH_INFO: `/v3/users/d263d7c1c7004ed7b26a0bc38bd98bdb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:43.978776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d263d7c1c7004ed7b26a0bc38bd98bdb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:43.980805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:44.290131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd263d7c1c7004ed7b26a0bc38bd98bdb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:44.291172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd263d7c1c7004ed7b26a0bc38bd98bdb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:44.299421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd263d7c1c7004ed7b26a0bc38bd98bdb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:44.300362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd263d7c1c7004ed7b26a0bc38bd98bdb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:44.403995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd263d7c1c7004ed7b26a0bc38bd98bdb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:44.422576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 896/1796] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:43 2022] DELETE /identity/v3/users/d263d7c1c7004ed7b26a0bc38bd98bdb => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:44.427744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39f70-da99-4fbf-b9e7-88dfadc87736 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:44.428248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39f70-da99-4fbf-b9e7-88dfadc87736 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:44.428345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39f70-da99-4fbf-b9e7-88dfadc87736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:44.448640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-eba39f70-da99-4fbf-b9e7-88dfadc87736 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:44.449844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eba39f70-da99-4fbf-b9e7-88dfadc87736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:44.516823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 901/1797] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:44.526840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:44.528244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:44.585903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22197ee-bfed-4ea7-b0f6-09b19733b491 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:44.589772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2e6e0-48dd-4825-93b4-74e22153c20c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:44.590369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2e6e0-48dd-4825-93b4-74e22153c20c service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:44.590645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2e6e0-48dd-4825-93b4-74e22153c20c service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:44.591030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2e6e0-48dd-4825-93b4-74e22153c20c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:44.604154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2e6e0-48dd-4825-93b4-74e22153c20c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:44.605612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2e6e0-48dd-4825-93b4-74e22153c20c service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:44.615165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 897/1798] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:45.061006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39f70-da99-4fbf-b9e7-88dfadc87736 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:45.062697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eba39f70-da99-4fbf-b9e7-88dfadc87736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:45.076503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39f70-da99-4fbf-b9e7-88dfadc87736 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:45.081616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:45.082275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.083225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.083498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] PATH_INFO: `/v3/projects/d5a0727ae3364629a8cdd35618e99f20` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.084632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a0727ae3364629a8cdd35618e99f20)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:45.087143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:45.211609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a0727ae3364629a8cdd35618e99f20', 'request_id': 'req-2a068701-1777-4fd7-bdb7-c9de8004dbd0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:45.340801 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a0727ae3364629a8cdd35618e99f20', 'request_id': 'req-2a068701-1777-4fd7-bdb7-c9de8004dbd0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:45.347021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a0727ae3364629a8cdd35618e99f20', 'request_id': 'req-2a068701-1777-4fd7-bdb7-c9de8004dbd0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:45.359013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] The token cache is being invalidate because project d5a0727ae3364629a8cdd35618e99f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:45.361981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:45.361981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2a068701-1777-4fd7-bdb7-c9de8004dbd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:45.361981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 902/1799] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:45 2022] DELETE /identity/v3/projects/d5a0727ae3364629a8cdd35618e99f20 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:45.367080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa0fe9-e983-4bd2-9ec2-789d820549a5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.367535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa0fe9-e983-4bd2-9ec2-789d820549a5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.367635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa0fe9-e983-4bd2-9ec2-789d820549a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.387103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-90fa0fe9-e983-4bd2-9ec2-789d820549a5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:45.388313 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-90fa0fe9-e983-4bd2-9ec2-789d820549a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:45.404843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:45.446730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 898/1800] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:45.459752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad991b6-bc6d-402a-964d-c36100c70241 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.459752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad991b6-bc6d-402a-964d-c36100c70241 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.459752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad991b6-bc6d-402a-964d-c36100c70241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.477941 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2ad991b6-bc6d-402a-964d-c36100c70241 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:45.479470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2ad991b6-bc6d-402a-964d-c36100c70241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:45.487999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 903/1801] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:45.492729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa0fe9-e983-4bd2-9ec2-789d820549a5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:45.502427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa0fe9-e983-4bd2-9ec2-789d820549a5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:45.508326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:45.508938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.511121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.511121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.511121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:45.511121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:45.512050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:45.524362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 899/1802] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:17:45.528995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad991b6-bc6d-402a-964d-c36100c70241 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:45.538926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad991b6-bc6d-402a-964d-c36100c70241 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:45.544370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375ca80-8da9-49ac-bef2-33cf1df058ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:45.545434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ca80-8da9-49ac-bef2-33cf1df058ba admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.545434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ca80-8da9-49ac-bef2-33cf1df058ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.545631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375ca80-8da9-49ac-bef2-33cf1df058ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.546151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375ca80-8da9-49ac-bef2-33cf1df058ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:45.548092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375ca80-8da9-49ac-bef2-33cf1df058ba admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:45.639990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 904/1803] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:45 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:45.646155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:45.656781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49289227-4d65-41b5-b2b3-b96a7f8f1612 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:45.716056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c99df-1f19-4f11-9d8c-56a621a433ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:45.716574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c99df-1f19-4f11-9d8c-56a621a433ba admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.716855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c99df-1f19-4f11-9d8c-56a621a433ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.717131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c99df-1f19-4f11-9d8c-56a621a433ba admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.717772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c99df-1f19-4f11-9d8c-56a621a433ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:45.719777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c99df-1f19-4f11-9d8c-56a621a433ba admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:45.822559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 900/1804] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:45 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:45.827131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375ca80-8da9-49ac-bef2-33cf1df058ba admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:45.837934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375ca80-8da9-49ac-bef2-33cf1df058ba admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:45.842498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297e80d-76a1-4083-94c3-82791a4f0aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:45.843185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297e80d-76a1-4083-94c3-82791a4f0aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.843463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297e80d-76a1-4083-94c3-82791a4f0aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.843734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297e80d-76a1-4083-94c3-82791a4f0aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.844372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297e80d-76a1-4083-94c3-82791a4f0aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:45.845962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297e80d-76a1-4083-94c3-82791a4f0aa8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:45.855783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 905/1805] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:45.859821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c99df-1f19-4f11-9d8c-56a621a433ba admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:45.871169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c99df-1f19-4f11-9d8c-56a621a433ba admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:45.876020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f53108-622e-49ed-a5fb-bc50365596e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:45.876714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f53108-622e-49ed-a5fb-bc50365596e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.877033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f53108-622e-49ed-a5fb-bc50365596e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.877368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f53108-622e-49ed-a5fb-bc50365596e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.878006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f53108-622e-49ed-a5fb-bc50365596e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:45.880364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f53108-622e-49ed-a5fb-bc50365596e1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:45.891907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 901/1806] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:45.896179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297e80d-76a1-4083-94c3-82791a4f0aa8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:45.910109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297e80d-76a1-4083-94c3-82791a4f0aa8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:45.914961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8664e42-b9e2-498f-930a-1d840f99b498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:45.915654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8664e42-b9e2-498f-930a-1d840f99b498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:45.915944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8664e42-b9e2-498f-930a-1d840f99b498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:45.916206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8664e42-b9e2-498f-930a-1d840f99b498 admin admin] PATH_INFO: `/v3/projects/e6f79d03d0e94da58f099cd7ce28d6b3/users/68ef093bb852441c95a21f74efc1e167/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:45.929431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:45.940113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8664e42-b9e2-498f-930a-1d840f99b498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f79d03d0e94da58f099cd7ce28d6b3, user_id=68ef093bb852441c95a21f74efc1e167, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:45.943556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8664e42-b9e2-498f-930a-1d840f99b498 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:46.039167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 906/1807] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:45 2022] PUT /identity/v3/projects/e6f79d03d0e94da58f099cd7ce28d6b3/users/68ef093bb852441c95a21f74efc1e167/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:46.050174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0596e-c31b-47e6-9e35-03910bd4b7e8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.050461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0596e-c31b-47e6-9e35-03910bd4b7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.050787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0596e-c31b-47e6-9e35-03910bd4b7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.079988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-99f0596e-c31b-47e6-9e35-03910bd4b7e8 None None] MFA Rules not processed for user `68ef093bb852441c95a21f74efc1e167`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:46.081601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-99f0596e-c31b-47e6-9e35-03910bd4b7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:46.146529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 902/1808] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:17:46.151264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8664e42-b9e2-498f-930a-1d840f99b498 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:46.214628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8664e42-b9e2-498f-930a-1d840f99b498 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:46.219723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417525b7-774c-4742-8081-123477442ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:46.220371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417525b7-774c-4742-8081-123477442ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.220602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417525b7-774c-4742-8081-123477442ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.220875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417525b7-774c-4742-8081-123477442ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.221491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417525b7-774c-4742-8081-123477442ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:46.223961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417525b7-774c-4742-8081-123477442ad1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:46.311822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 907/1809] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:46 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:46.315921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0596e-c31b-47e6-9e35-03910bd4b7e8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:46.326641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0596e-c31b-47e6-9e35-03910bd4b7e8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:46.339707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:46.386332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024622-c81c-4ee0-b807-9115a12ae41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:46.387388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024622-c81c-4ee0-b807-9115a12ae41d admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.387725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024622-c81c-4ee0-b807-9115a12ae41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.395739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024622-c81c-4ee0-b807-9115a12ae41d admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.395739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024622-c81c-4ee0-b807-9115a12ae41d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:46.395739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024622-c81c-4ee0-b807-9115a12ae41d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:46.496859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 903/1810] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:46 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:46.501358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417525b7-774c-4742-8081-123477442ad1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:46.511983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417525b7-774c-4742-8081-123477442ad1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:46.516559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca1ce-fd99-4757-949f-0879b54eeaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:46.517152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca1ce-fd99-4757-949f-0879b54eeaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.517499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca1ce-fd99-4757-949f-0879b54eeaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.517777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca1ce-fd99-4757-949f-0879b54eeaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.518414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ca1ce-fd99-4757-949f-0879b54eeaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:46.520057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ca1ce-fd99-4757-949f-0879b54eeaa6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:46.531155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 908/1811] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:46.535228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024622-c81c-4ee0-b807-9115a12ae41d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:46.546250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024622-c81c-4ee0-b807-9115a12ae41d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:46.551108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2bf06-c15d-4925-b4c0-793f3f29ca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:46.551858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2bf06-c15d-4925-b4c0-793f3f29ca13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.552138 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2bf06-c15d-4925-b4c0-793f3f29ca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.552414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2bf06-c15d-4925-b4c0-793f3f29ca13 admin admin] PATH_INFO: `/v3/projects/bf9881e5ba2c48d5878cb1751c7a1f7e/users/8b7702f5e20b4640a8827af321098f04/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.567280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2bf06-c15d-4925-b4c0-793f3f29ca13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9881e5ba2c48d5878cb1751c7a1f7e, user_id=8b7702f5e20b4640a8827af321098f04, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:46.570167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2bf06-c15d-4925-b4c0-793f3f29ca13 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:46.659089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 904/1812] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:46 2022] PUT /identity/v3/projects/bf9881e5ba2c48d5878cb1751c7a1f7e/users/8b7702f5e20b4640a8827af321098f04/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:46.667287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca1ce-fd99-4757-949f-0879b54eeaa6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:46.687379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca1ce-fd99-4757-949f-0879b54eeaa6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:46.737373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d1c9-0dc2-474d-a816-13fdc470df4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:46.737928 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d1c9-0dc2-474d-a816-13fdc470df4a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.738207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d1c9-0dc2-474d-a816-13fdc470df4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.738481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d1c9-0dc2-474d-a816-13fdc470df4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.739157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00d1c9-0dc2-474d-a816-13fdc470df4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:46.740790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00d1c9-0dc2-474d-a816-13fdc470df4a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:46.750923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 909/1813] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:46.755186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2bf06-c15d-4925-b4c0-793f3f29ca13 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:46.765682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2bf06-c15d-4925-b4c0-793f3f29ca13 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:46.771121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:46.771773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.772051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.772332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] PATH_INFO: `/v3/projects/bf9881e5ba2c48d5878cb1751c7a1f7e/users/8b7702f5e20b4640a8827af321098f04/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.774245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9881e5ba2c48d5878cb1751c7a1f7e, user_id=8b7702f5e20b4640a8827af321098f04, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:46.777666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:46.790064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 905/1814] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:46 2022] PUT /identity/v3/projects/bf9881e5ba2c48d5878cb1751c7a1f7e/users/8b7702f5e20b4640a8827af321098f04/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:46.802273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8523e7-0789-4810-9d5b-b36e0f2cbad7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.802506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8523e7-0789-4810-9d5b-b36e0f2cbad7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.802745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8523e7-0789-4810-9d5b-b36e0f2cbad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.831022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4e8523e7-0789-4810-9d5b-b36e0f2cbad7 None None] MFA Rules not processed for user `8b7702f5e20b4640a8827af321098f04`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:46.832212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4e8523e7-0789-4810-9d5b-b36e0f2cbad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:46.879305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:46.901461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 910/1815] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:46.912878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:46.914625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:46.972678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccc9a8-185e-4322-9ab0-5c68bf119b1e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:46.977105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa7e51-f2b1-47f3-bacd-c32c1f214f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:46.977720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7e51-f2b1-47f3-bacd-c32c1f214f72 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:46.978002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7e51-f2b1-47f3-bacd-c32c1f214f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:46.978580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7e51-f2b1-47f3-bacd-c32c1f214f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:46.988411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa7e51-f2b1-47f3-bacd-c32c1f214f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:46.990422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa7e51-f2b1-47f3-bacd-c32c1f214f72 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:47.000439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 906/1816] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:48.033577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8523e7-0789-4810-9d5b-b36e0f2cbad7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:48.096767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8523e7-0789-4810-9d5b-b36e0f2cbad7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:48.101577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:48.102200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:48.102473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:48.102739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] PATH_INFO: `/v3/users/68ef093bb852441c95a21f74efc1e167` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:48.103925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ef093bb852441c95a21f74efc1e167)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:48.107731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:48.327748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ef093bb852441c95a21f74efc1e167'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:48.337027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ef093bb852441c95a21f74efc1e167'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:48.337726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ef093bb852441c95a21f74efc1e167'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:48.338571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ef093bb852441c95a21f74efc1e167'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:48.352626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ef093bb852441c95a21f74efc1e167'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:48.425991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 911/1817] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:48 2022] DELETE /identity/v3/users/68ef093bb852441c95a21f74efc1e167 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:48.434327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a9c97-a2ec-4b6b-bbf1-3720b26b1df4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:48.434720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a9c97-a2ec-4b6b-bbf1-3720b26b1df4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:48.435044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a9c97-a2ec-4b6b-bbf1-3720b26b1df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:48.457023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6d4a9c97-a2ec-4b6b-bbf1-3720b26b1df4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:48.459037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6d4a9c97-a2ec-4b6b-bbf1-3720b26b1df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:48.523321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 907/1818] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:48.533718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:48.535674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:48.598350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42dff4-4d54-4e63-897c-d1243da81765 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:48.602337 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd35a83-9424-4827-b028-14e1d8db0086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:48.603069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd35a83-9424-4827-b028-14e1d8db0086 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:48.603349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd35a83-9424-4827-b028-14e1d8db0086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:48.603681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd35a83-9424-4827-b028-14e1d8db0086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:48.614164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd35a83-9424-4827-b028-14e1d8db0086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:48.615715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd35a83-9424-4827-b028-14e1d8db0086 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:48.626485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 912/1819] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:50.303599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d731ae-ba3c-4f78-8157-e9afbc5b69cb None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:50.304131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d731ae-ba3c-4f78-8157-e9afbc5b69cb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:50.304279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d731ae-ba3c-4f78-8157-e9afbc5b69cb None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:50.305876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 908/1820] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:17:50.316083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8e04-b897-439e-8ad0-02e8578917d7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:50.316083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8e04-b897-439e-8ad0-02e8578917d7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:50.316083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f8e04-b897-439e-8ad0-02e8578917d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:50.329698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:50.336432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-eb9f8e04-b897-439e-8ad0-02e8578917d7 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:50.337756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eb9f8e04-b897-439e-8ad0-02e8578917d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:50.362689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d731ae-ba3c-4f78-8157-e9afbc5b69cb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:50.366006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-13d731ae-ba3c-4f78-8157-e9afbc5b69cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:50.386526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d731ae-ba3c-4f78-8157-e9afbc5b69cb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:50.392388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:50.396647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:50.396647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:50.396647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] PATH_INFO: `/v3/projects/e6f79d03d0e94da58f099cd7ce28d6b3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:50.396647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f79d03d0e94da58f099cd7ce28d6b3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:50.396971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:50.410684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:50.425962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 913/1821] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:17:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:50.440491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f8e04-b897-439e-8ad0-02e8578917d7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:50.442584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eb9f8e04-b897-439e-8ad0-02e8578917d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:50.456227 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f8e04-b897-439e-8ad0-02e8578917d7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:50.462333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94175f81-30da-4fe5-ba62-46a7b6654161 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:50.462933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175f81-30da-4fe5-ba62-46a7b6654161 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:50.463234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175f81-30da-4fe5-ba62-46a7b6654161 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:50.463508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94175f81-30da-4fe5-ba62-46a7b6654161 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:50.474518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94175f81-30da-4fe5-ba62-46a7b6654161 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:50.478628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94175f81-30da-4fe5-ba62-46a7b6654161 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:50.486771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 914/1822] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:17:50.585200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f79d03d0e94da58f099cd7ce28d6b3', 'request_id': 'req-e97606f8-6101-4c1a-861b-6ed4286e698a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:50.591079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f79d03d0e94da58f099cd7ce28d6b3', 'request_id': 'req-e97606f8-6101-4c1a-861b-6ed4286e698a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:50.598618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f79d03d0e94da58f099cd7ce28d6b3', 'request_id': 'req-e97606f8-6101-4c1a-861b-6ed4286e698a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:50.737709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] The token cache is being invalidate because project e6f79d03d0e94da58f099cd7ce28d6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:50.738267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:50.749785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:50.752994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 909/1823] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:50 2022] DELETE /identity/v3/projects/e6f79d03d0e94da58f099cd7ce28d6b3 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:50.757731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94175f81-30da-4fe5-ba62-46a7b6654161 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:50.771886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94175f81-30da-4fe5-ba62-46a7b6654161 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:50.844554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:50.845191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:50.845471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:50.845742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] PATH_INFO: `/v3/users/8b7702f5e20b4640a8827af321098f04` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:50.846873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7702f5e20b4640a8827af321098f04)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:50.849382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:50.997639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7702f5e20b4640a8827af321098f04'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.004979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7702f5e20b4640a8827af321098f04'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.005705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7702f5e20b4640a8827af321098f04'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.006326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7702f5e20b4640a8827af321098f04'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.018327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8f94386-fdb4-44d9-8806-6258fa09d421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7702f5e20b4640a8827af321098f04'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.103288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 915/1824] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:50 2022] DELETE /identity/v3/users/8b7702f5e20b4640a8827af321098f04 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:51.663796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:51.665257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:51.679006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97606f8-6101-4c1a-861b-6ed4286e698a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:51.737377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:51.738000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:51.738276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:51.738545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] PATH_INFO: `/v3/projects/bf9881e5ba2c48d5878cb1751c7a1f7e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:51.739688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9881e5ba2c48d5878cb1751c7a1f7e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:51.741583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:51.866641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9881e5ba2c48d5878cb1751c7a1f7e', 'request_id': 'req-a91fbcc0-983c-469f-8efb-d9dfc3c35609'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.872650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9881e5ba2c48d5878cb1751c7a1f7e', 'request_id': 'req-a91fbcc0-983c-469f-8efb-d9dfc3c35609'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.878951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9881e5ba2c48d5878cb1751c7a1f7e', 'request_id': 'req-a91fbcc0-983c-469f-8efb-d9dfc3c35609'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.953307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] The token cache is being invalidate because project bf9881e5ba2c48d5878cb1751c7a1f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:51.953599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.954243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a91fbcc0-983c-469f-8efb-d9dfc3c35609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:51.955838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 910/1825] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:51 2022] DELETE /identity/v3/projects/bf9881e5ba2c48d5878cb1751c7a1f7e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:51.961232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182a08c-0dc5-412e-8bdc-4b6bb00f38a9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:51.961663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182a08c-0dc5-412e-8bdc-4b6bb00f38a9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:51.961663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182a08c-0dc5-412e-8bdc-4b6bb00f38a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:51.971162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:51.982534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2182a08c-0dc5-412e-8bdc-4b6bb00f38a9 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:51.983832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2182a08c-0dc5-412e-8bdc-4b6bb00f38a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:52.042966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 916/1826] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:52.050704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c173ec-1919-46a9-9bd1-719eaf168aa8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.051036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c173ec-1919-46a9-9bd1-719eaf168aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.051316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c173ec-1919-46a9-9bd1-719eaf168aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.070656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b3c173ec-1919-46a9-9bd1-719eaf168aa8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:52.071975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b3c173ec-1919-46a9-9bd1-719eaf168aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:52.080472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 911/1827] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:52.086962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182a08c-0dc5-412e-8bdc-4b6bb00f38a9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:52.097261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182a08c-0dc5-412e-8bdc-4b6bb00f38a9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:52.102388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:52.103020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.103305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.103603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.104257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:52.104562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:52.106200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:52.118362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 917/1828] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:17:52.122670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c173ec-1919-46a9-9bd1-719eaf168aa8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:52.132913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c173ec-1919-46a9-9bd1-719eaf168aa8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:52.137976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae65c1-89c8-4394-9e55-c933e6b37aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:52.138579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae65c1-89c8-4394-9e55-c933e6b37aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.138891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae65c1-89c8-4394-9e55-c933e6b37aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.139201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae65c1-89c8-4394-9e55-c933e6b37aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.139832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae65c1-89c8-4394-9e55-c933e6b37aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:52.141897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae65c1-89c8-4394-9e55-c933e6b37aa9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:52.268503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:52.276522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 912/1829] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:52 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:52.281160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:52.293442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a500a-ab79-4b74-a5e6-7d1a11adad78 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:52.358965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a5fcd-ad28-4348-ab00-4239e2c2f91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:52.359572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a5fcd-ad28-4348-ab00-4239e2c2f91f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.359850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a5fcd-ad28-4348-ab00-4239e2c2f91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.360265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a5fcd-ad28-4348-ab00-4239e2c2f91f admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.360927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a5fcd-ad28-4348-ab00-4239e2c2f91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:52.362919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a5fcd-ad28-4348-ab00-4239e2c2f91f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:52.497299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 918/1830] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:52 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:52.503458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae65c1-89c8-4394-9e55-c933e6b37aa9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:52.514649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae65c1-89c8-4394-9e55-c933e6b37aa9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:52.519310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efca545-3223-4e3d-bef7-e6a638ee5e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:52.519932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efca545-3223-4e3d-bef7-e6a638ee5e7f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.520210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efca545-3223-4e3d-bef7-e6a638ee5e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.520477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efca545-3223-4e3d-bef7-e6a638ee5e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.521120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efca545-3223-4e3d-bef7-e6a638ee5e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:52.522708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efca545-3223-4e3d-bef7-e6a638ee5e7f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:52.536587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 913/1831] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:52.538268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a5fcd-ad28-4348-ab00-4239e2c2f91f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:52.548821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a5fcd-ad28-4348-ab00-4239e2c2f91f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:52.554189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b496165-15c0-4de7-9651-628f844ee2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:52.554788 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496165-15c0-4de7-9651-628f844ee2ae admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.555140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496165-15c0-4de7-9651-628f844ee2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.555415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496165-15c0-4de7-9651-628f844ee2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.556052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496165-15c0-4de7-9651-628f844ee2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:52.557737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496165-15c0-4de7-9651-628f844ee2ae admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:52.570248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 919/1832] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:52.574482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efca545-3223-4e3d-bef7-e6a638ee5e7f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:52.584860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efca545-3223-4e3d-bef7-e6a638ee5e7f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:52.589512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ecf0-6d1b-436c-9df0-7438d98596a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:52.590167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ecf0-6d1b-436c-9df0-7438d98596a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.590450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ecf0-6d1b-436c-9df0-7438d98596a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.590725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ecf0-6d1b-436c-9df0-7438d98596a6 admin admin] PATH_INFO: `/v3/projects/c8fa2a8460b041c4a72e57c2ef35bc8d/users/4e7b70887d8d4bbc8c30a2ba260bdf8f/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.605540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69ecf0-6d1b-436c-9df0-7438d98596a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8fa2a8460b041c4a72e57c2ef35bc8d, user_id=4e7b70887d8d4bbc8c30a2ba260bdf8f, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:52.608318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69ecf0-6d1b-436c-9df0-7438d98596a6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:52.686432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 914/1833] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:52 2022] PUT /identity/v3/projects/c8fa2a8460b041c4a72e57c2ef35bc8d/users/4e7b70887d8d4bbc8c30a2ba260bdf8f/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:52.696934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219b805-b47d-4925-a908-ad203d31dd7e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.697217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219b805-b47d-4925-a908-ad203d31dd7e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.697485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219b805-b47d-4925-a908-ad203d31dd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.731454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7219b805-b47d-4925-a908-ad203d31dd7e None None] MFA Rules not processed for user `4e7b70887d8d4bbc8c30a2ba260bdf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:52.732724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7219b805-b47d-4925-a908-ad203d31dd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:52.797321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:52.800853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 920/1834] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:52.805384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ecf0-6d1b-436c-9df0-7438d98596a6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:52.866330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ecf0-6d1b-436c-9df0-7438d98596a6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:52.870906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57547500-e9c9-4a47-b00f-45e7683888c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:52.871506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57547500-e9c9-4a47-b00f-45e7683888c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.871781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57547500-e9c9-4a47-b00f-45e7683888c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.872047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57547500-e9c9-4a47-b00f-45e7683888c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.872671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57547500-e9c9-4a47-b00f-45e7683888c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:52.874679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57547500-e9c9-4a47-b00f-45e7683888c1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:52.942106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219b805-b47d-4925-a908-ad203d31dd7e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:52.944129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7219b805-b47d-4925-a908-ad203d31dd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:52.957980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219b805-b47d-4925-a908-ad203d31dd7e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:52.963367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:52.963991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:52.964305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:52.964575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] PATH_INFO: `/v3/users/6bf59213ab0648c996a1da73964afea8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:52.965710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bf59213ab0648c996a1da73964afea8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:52.967680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:53.013467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 915/1835] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:52 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:53.018319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57547500-e9c9-4a47-b00f-45e7683888c1 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:53.128697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57547500-e9c9-4a47-b00f-45e7683888c1 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:53.136517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:53.138112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.138406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.138686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.139364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:53.141664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:53.237688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bf59213ab0648c996a1da73964afea8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:53.245921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bf59213ab0648c996a1da73964afea8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:53.246687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bf59213ab0648c996a1da73964afea8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:53.247316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bf59213ab0648c996a1da73964afea8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:53.260185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3dd0a55a-4a3a-48ac-8795-5b377523b469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bf59213ab0648c996a1da73964afea8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:53.290085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 916/1836] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:53 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:53.294626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:53.296679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:53.310104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c1d89-ebcc-4009-8c92-419ed5cf7a0c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:53.314876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9707e89-e04c-442f-a0a4-e759900f7e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:53.315621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9707e89-e04c-442f-a0a4-e759900f7e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.318923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9707e89-e04c-442f-a0a4-e759900f7e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.318923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9707e89-e04c-442f-a0a4-e759900f7e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.318923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9707e89-e04c-442f-a0a4-e759900f7e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:53.319270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9707e89-e04c-442f-a0a4-e759900f7e06 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:53.330868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 917/1837] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:53.335526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9707e89-e04c-442f-a0a4-e759900f7e06 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:53.345105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9707e89-e04c-442f-a0a4-e759900f7e06 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:53.350514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11d3a-5df7-460c-bf44-f89a370e18bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:53.351202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11d3a-5df7-460c-bf44-f89a370e18bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.351484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11d3a-5df7-460c-bf44-f89a370e18bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.351759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11d3a-5df7-460c-bf44-f89a370e18bf admin admin] PATH_INFO: `/v3/projects/e0ccce3cce0141478ac5b1e45acd0cfa/users/95673ebd60e94d5faa6e46866d94fbba/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.366537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d11d3a-5df7-460c-bf44-f89a370e18bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ccce3cce0141478ac5b1e45acd0cfa, user_id=95673ebd60e94d5faa6e46866d94fbba, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:53.369225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d11d3a-5df7-460c-bf44-f89a370e18bf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:53.399605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 921/1838] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:52 2022] DELETE /identity/v3/users/6bf59213ab0648c996a1da73964afea8 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:53.404122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d604cf-b118-4a3b-af2a-128e7f4311e5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.404388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d604cf-b118-4a3b-af2a-128e7f4311e5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.404692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d604cf-b118-4a3b-af2a-128e7f4311e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.427469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-75d604cf-b118-4a3b-af2a-128e7f4311e5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:53.428731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-75d604cf-b118-4a3b-af2a-128e7f4311e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:53.491717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 922/1839] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:17:53.500545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 918/1840] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:53 2022] PUT /identity/v3/projects/e0ccce3cce0141478ac5b1e45acd0cfa/users/95673ebd60e94d5faa6e46866d94fbba/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:53.506837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d604cf-b118-4a3b-af2a-128e7f4311e5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:53.506837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-75d604cf-b118-4a3b-af2a-128e7f4311e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:53.509800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11d3a-5df7-460c-bf44-f89a370e18bf admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:53.514495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:53.580093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d604cf-b118-4a3b-af2a-128e7f4311e5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:53.587479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11d3a-5df7-460c-bf44-f89a370e18bf admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:53.592136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe7039-a681-4613-b2f6-b7d06aa5f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:53.592743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7039-a681-4613-b2f6-b7d06aa5f133 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.593140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7039-a681-4613-b2f6-b7d06aa5f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.593385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe7039-a681-4613-b2f6-b7d06aa5f133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.594029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe7039-a681-4613-b2f6-b7d06aa5f133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:53.595661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe7039-a681-4613-b2f6-b7d06aa5f133 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:53.605528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 919/1841] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:53.608145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:53.609611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe7039-a681-4613-b2f6-b7d06aa5f133 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:53.619301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe7039-a681-4613-b2f6-b7d06aa5f133 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:53.624088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc8a59-f9c9-487b-9bf2-5a1fe3566858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:53.624737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc8a59-f9c9-487b-9bf2-5a1fe3566858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.625020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc8a59-f9c9-487b-9bf2-5a1fe3566858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.625293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc8a59-f9c9-487b-9bf2-5a1fe3566858 admin admin] PATH_INFO: `/v3/projects/e0ccce3cce0141478ac5b1e45acd0cfa/users/95673ebd60e94d5faa6e46866d94fbba/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.627222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc8a59-f9c9-487b-9bf2-5a1fe3566858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ccce3cce0141478ac5b1e45acd0cfa, user_id=95673ebd60e94d5faa6e46866d94fbba, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:53.629880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc8a59-f9c9-487b-9bf2-5a1fe3566858 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:53.632683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:53.633289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.633880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.633988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.651173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 920/1842] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:53 2022] PUT /identity/v3/projects/e0ccce3cce0141478ac5b1e45acd0cfa/users/95673ebd60e94d5faa6e46866d94fbba/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:17:53.651902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:53.651902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:53.653790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55273-a6d2-4019-b2f4-a272576b4e3c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.654080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55273-a6d2-4019-b2f4-a272576b4e3c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.654357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55273-a6d2-4019-b2f4-a272576b4e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.657404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 923/1843] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:53.684463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ffe55273-a6d2-4019-b2f4-a272576b4e3c None None] MFA Rules not processed for user `95673ebd60e94d5faa6e46866d94fbba`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:53.685730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ffe55273-a6d2-4019-b2f4-a272576b4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:53.754782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 921/1844] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:53.768440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:53.770167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:53.841232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1970b35-a6ca-42eb-87bd-71cd7c46ceda service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:53.895211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe2c63-6622-46d7-8141-1ade22eb6fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:53.895942 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe2c63-6622-46d7-8141-1ade22eb6fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:53.896234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe2c63-6622-46d7-8141-1ade22eb6fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:53.896510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe2c63-6622-46d7-8141-1ade22eb6fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:53.905798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe2c63-6622-46d7-8141-1ade22eb6fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:53.907423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe2c63-6622-46d7-8141-1ade22eb6fb1 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:53.918376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 924/1845] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:54.175659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe55273-a6d2-4019-b2f4-a272576b4e3c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:54.177565 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ffe55273-a6d2-4019-b2f4-a272576b4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:54.232954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe2c63-6622-46d7-8141-1ade22eb6fb1 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:54.242681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe2c63-6622-46d7-8141-1ade22eb6fb1 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:54.248219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:54.252563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:54.252563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:54.252563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:54.252563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:54.270531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe55273-a6d2-4019-b2f4-a272576b4e3c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:54.276854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:54.277489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:54.277765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:54.278031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] PATH_INFO: `/v3/projects/231e7186d1fd4a89aac442db1e8db47f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:54.279260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231e7186d1fd4a89aac442db1e8db47f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:54.281240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:54.294870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:54.324437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:54.327090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:54.335850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 925/1846] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:17:54.450570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231e7186d1fd4a89aac442db1e8db47f', 'request_id': 'req-2b9b2428-44ea-4fce-bc4b-af4a50eec118'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.456169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231e7186d1fd4a89aac442db1e8db47f', 'request_id': 'req-2b9b2428-44ea-4fce-bc4b-af4a50eec118'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.462197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231e7186d1fd4a89aac442db1e8db47f', 'request_id': 'req-2b9b2428-44ea-4fce-bc4b-af4a50eec118'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.569784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] The token cache is being invalidate because project 231e7186d1fd4a89aac442db1e8db47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:54.570102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.570745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.572264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 922/1847] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:54 2022] DELETE /identity/v3/projects/231e7186d1fd4a89aac442db1e8db47f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:54.576401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:54.587791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6753bd-5386-4a1c-a6b2-e7ad46f3c475 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:54.616052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:54.646738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:54.647514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:54.647948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:54.648323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] PATH_INFO: `/v3/users/ed51541ab3f04a7491b07201007f2a9f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:54.649702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed51541ab3f04a7491b07201007f2a9f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:54.651743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:54.849735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed51541ab3f04a7491b07201007f2a9f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.858131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed51541ab3f04a7491b07201007f2a9f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.859134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed51541ab3f04a7491b07201007f2a9f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.859921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed51541ab3f04a7491b07201007f2a9f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.874474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2b32aba5-c206-4d5d-b0ed-fdf4ab53577d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed51541ab3f04a7491b07201007f2a9f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:54.975412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 926/1848] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:17:54 2022] DELETE /identity/v3/users/ed51541ab3f04a7491b07201007f2a9f => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:55.464476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:55.466005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:55.483730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b2428-44ea-4fce-bc4b-af4a50eec118 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:55.544394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:55.545035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:55.545321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:55.545594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] PATH_INFO: `/v3/projects/c997742a305149afa395083a4eddce31` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:55.546725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c997742a305149afa395083a4eddce31)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:55.548666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:55.693964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c997742a305149afa395083a4eddce31', 'request_id': 'req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:55.699580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c997742a305149afa395083a4eddce31', 'request_id': 'req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:55.706260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c997742a305149afa395083a4eddce31', 'request_id': 'req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:55.827580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] The token cache is being invalidate because project c997742a305149afa395083a4eddce31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:17:55.827868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:55.828751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e694e8ff-aaa0-4648-87a0-542dee4e9f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:17:55.830409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 923/1849] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:17:55 2022] DELETE /identity/v3/projects/c997742a305149afa395083a4eddce31 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:55.835793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72a46e-e652-4202-9170-74846c8b1301 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:55.836315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72a46e-e652-4202-9170-74846c8b1301 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:55.836315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72a46e-e652-4202-9170-74846c8b1301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:55.856724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dd72a46e-e652-4202-9170-74846c8b1301 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:55.857941 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dd72a46e-e652-4202-9170-74846c8b1301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:55.925836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 927/1850] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:55.933660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af1059-f2e6-43d7-8a6e-5c0b93356777 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:55.933918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af1059-f2e6-43d7-8a6e-5c0b93356777 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:55.934183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af1059-f2e6-43d7-8a6e-5c0b93356777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:55.954621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d8af1059-f2e6-43d7-8a6e-5c0b93356777 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:55.955889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d8af1059-f2e6-43d7-8a6e-5c0b93356777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:55.964573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 924/1851] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:55.968935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72a46e-e652-4202-9170-74846c8b1301 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:55.978651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72a46e-e652-4202-9170-74846c8b1301 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:55.983479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:55.984115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:55.984392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:55.984653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:55.985286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:55.985581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:55.987402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:56.000248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 928/1852] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:17:56.007393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af1059-f2e6-43d7-8a6e-5c0b93356777 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:56.017306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af1059-f2e6-43d7-8a6e-5c0b93356777 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:56.023054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fa1d7-f6b9-4f0a-9dd4-0d176454af15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:56.023660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa1d7-f6b9-4f0a-9dd4-0d176454af15 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:56.023938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa1d7-f6b9-4f0a-9dd4-0d176454af15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:56.024211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa1d7-f6b9-4f0a-9dd4-0d176454af15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:56.024843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086fa1d7-f6b9-4f0a-9dd4-0d176454af15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:56.027138 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086fa1d7-f6b9-4f0a-9dd4-0d176454af15 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:56.126597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 925/1853] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:56 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:56.131455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:56.141807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ef045-499b-4ca9-864d-ec68013cdc4c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:56.174620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:56.202655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95744dd-b7e7-4936-8615-99f8a5eae21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:56.203559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95744dd-b7e7-4936-8615-99f8a5eae21c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:56.203841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95744dd-b7e7-4936-8615-99f8a5eae21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:56.204142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95744dd-b7e7-4936-8615-99f8a5eae21c admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:56.204724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95744dd-b7e7-4936-8615-99f8a5eae21c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:56.206681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95744dd-b7e7-4936-8615-99f8a5eae21c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:56.324880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 929/1854] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:56 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:56.329334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fa1d7-f6b9-4f0a-9dd4-0d176454af15 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:56.335973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:56.344464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fa1d7-f6b9-4f0a-9dd4-0d176454af15 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:56.350044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a190f-c30d-445e-95b1-9940b0058c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:56.350658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a190f-c30d-445e-95b1-9940b0058c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:56.351130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a190f-c30d-445e-95b1-9940b0058c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:56.351405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a190f-c30d-445e-95b1-9940b0058c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:56.352231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a190f-c30d-445e-95b1-9940b0058c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:56.354030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a190f-c30d-445e-95b1-9940b0058c09 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:56.369913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 926/1855] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:17:56.371027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95744dd-b7e7-4936-8615-99f8a5eae21c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:56.387167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95744dd-b7e7-4936-8615-99f8a5eae21c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:56.393074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb24647-ea11-4d56-8a15-ed5b19141df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:56.394304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24647-ea11-4d56-8a15-ed5b19141df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:56.394304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24647-ea11-4d56-8a15-ed5b19141df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:56.394636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24647-ea11-4d56-8a15-ed5b19141df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:56.395278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb24647-ea11-4d56-8a15-ed5b19141df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:56.396985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb24647-ea11-4d56-8a15-ed5b19141df2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:56.408939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 930/1856] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:56.413548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a190f-c30d-445e-95b1-9940b0058c09 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:56.433720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a190f-c30d-445e-95b1-9940b0058c09 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:56.438785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867544c6-6128-4c93-8f87-51cd5cf0646c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:56.439503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867544c6-6128-4c93-8f87-51cd5cf0646c admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:56.439781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867544c6-6128-4c93-8f87-51cd5cf0646c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:56.440051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867544c6-6128-4c93-8f87-51cd5cf0646c admin admin] PATH_INFO: `/v3/projects/b3caa0978fa741ef89acfb625fc5d66a/users/1625d2a762604d8e9b672d0258e069fb/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:56.460671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867544c6-6128-4c93-8f87-51cd5cf0646c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3caa0978fa741ef89acfb625fc5d66a, user_id=1625d2a762604d8e9b672d0258e069fb, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:56.460671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867544c6-6128-4c93-8f87-51cd5cf0646c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:56.608515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 927/1857] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:56 2022] PUT /identity/v3/projects/b3caa0978fa741ef89acfb625fc5d66a/users/1625d2a762604d8e9b672d0258e069fb/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:56.619804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0047e40-89c6-45e6-b322-31853f660378 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:56.620170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0047e40-89c6-45e6-b322-31853f660378 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:56.620506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0047e40-89c6-45e6-b322-31853f660378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:56.656317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a0047e40-89c6-45e6-b322-31853f660378 None None] MFA Rules not processed for user `1625d2a762604d8e9b672d0258e069fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:56.664423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a0047e40-89c6-45e6-b322-31853f660378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:56.767385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 931/1858] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:56.771964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867544c6-6128-4c93-8f87-51cd5cf0646c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:56.837229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867544c6-6128-4c93-8f87-51cd5cf0646c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:56.842000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547c67-b7dc-44f0-9d08-3396a872a704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:56.842608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547c67-b7dc-44f0-9d08-3396a872a704 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:56.842966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547c67-b7dc-44f0-9d08-3396a872a704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:56.843193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547c67-b7dc-44f0-9d08-3396a872a704 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:56.843935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d547c67-b7dc-44f0-9d08-3396a872a704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:56.845794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d547c67-b7dc-44f0-9d08-3396a872a704 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:57.200268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 928/1859] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:56 2022] POST /identity/v3/projects => generated 381 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:17:57.204688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0047e40-89c6-45e6-b322-31853f660378 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:57.216265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0047e40-89c6-45e6-b322-31853f660378 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:57.275007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcccc8-1e91-478e-9ba4-6fbb626c7fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:57.275558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcccc8-1e91-478e-9ba4-6fbb626c7fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:57.275841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcccc8-1e91-478e-9ba4-6fbb626c7fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:57.276115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcccc8-1e91-478e-9ba4-6fbb626c7fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:57.276738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcccc8-1e91-478e-9ba4-6fbb626c7fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:57.278789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcccc8-1e91-478e-9ba4-6fbb626c7fa4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:57.515121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 932/1860] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:17:57 2022] POST /identity/v3/users => generated 297 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:57.519017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547c67-b7dc-44f0-9d08-3396a872a704 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:57.530105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547c67-b7dc-44f0-9d08-3396a872a704 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:57.534626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5eca7-cb73-4aed-94a9-41ccb021e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:57.535258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5eca7-cb73-4aed-94a9-41ccb021e439 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:57.535538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5eca7-cb73-4aed-94a9-41ccb021e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:57.535810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5eca7-cb73-4aed-94a9-41ccb021e439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:57.536453 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5eca7-cb73-4aed-94a9-41ccb021e439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:57.538594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5eca7-cb73-4aed-94a9-41ccb021e439 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:57.548742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 929/1861] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:57.553113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcccc8-1e91-478e-9ba4-6fbb626c7fa4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:57.563410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcccc8-1e91-478e-9ba4-6fbb626c7fa4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:57.568684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f7ee9-b965-43b5-a87d-52b83d52c1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:57.569330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7ee9-b965-43b5-a87d-52b83d52c1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:57.569604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7ee9-b965-43b5-a87d-52b83d52c1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:57.569876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7ee9-b965-43b5-a87d-52b83d52c1c8 admin admin] PATH_INFO: `/v3/projects/16cf2957975e4b6cbee19d683bf843d0/users/e8c9856440cc4b36a07a647aa3ba087f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:57.589035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f7ee9-b965-43b5-a87d-52b83d52c1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cf2957975e4b6cbee19d683bf843d0, user_id=e8c9856440cc4b36a07a647aa3ba087f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:57.592367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f7ee9-b965-43b5-a87d-52b83d52c1c8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:57.794786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 933/1862] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:57 2022] PUT /identity/v3/projects/16cf2957975e4b6cbee19d683bf843d0/users/e8c9856440cc4b36a07a647aa3ba087f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:57.798536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5eca7-cb73-4aed-94a9-41ccb021e439 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:57.815902 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5eca7-cb73-4aed-94a9-41ccb021e439 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:57.923668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ec4a0-5acc-4c19-bd08-50735a8570e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:57.924637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec4a0-5acc-4c19-bd08-50735a8570e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:57.925187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec4a0-5acc-4c19-bd08-50735a8570e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:57.925589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec4a0-5acc-4c19-bd08-50735a8570e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:57.926724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ec4a0-5acc-4c19-bd08-50735a8570e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:57.929299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ec4a0-5acc-4c19-bd08-50735a8570e5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:57.951350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 930/1863] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:17:57.951769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f7ee9-b965-43b5-a87d-52b83d52c1c8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:57.963323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f7ee9-b965-43b5-a87d-52b83d52c1c8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:57.969021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:57.969850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:57.970196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:57.970968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] PATH_INFO: `/v3/projects/16cf2957975e4b6cbee19d683bf843d0/users/e8c9856440cc4b36a07a647aa3ba087f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:57.973250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cf2957975e4b6cbee19d683bf843d0, user_id=e8c9856440cc4b36a07a647aa3ba087f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:57.976262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:57.981538 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:57.994913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 934/1864] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:17:57 2022] PUT /identity/v3/projects/16cf2957975e4b6cbee19d683bf843d0/users/e8c9856440cc4b36a07a647aa3ba087f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:17:58.007213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a4ff-5ad7-48cc-a3f3-b72003d5c184 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:58.007600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a4ff-5ad7-48cc-a3f3-b72003d5c184 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:58.007765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486a4ff-5ad7-48cc-a3f3-b72003d5c184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:58.045652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3486a4ff-5ad7-48cc-a3f3-b72003d5c184 None None] MFA Rules not processed for user `e8c9856440cc4b36a07a647aa3ba087f`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:17:58.046825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3486a4ff-5ad7-48cc-a3f3-b72003d5c184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:58.083875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:17:58.179016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 931/1865] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:17:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:17:58.198934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:58.200979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:17:58.265894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8c245-ec84-4004-84da-d4a823cd7f2b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:58.272375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5e68-1553-432b-b716-2dd951c3fd25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:58.273123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5e68-1553-432b-b716-2dd951c3fd25 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:58.273476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5e68-1553-432b-b716-2dd951c3fd25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:58.273758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5e68-1553-432b-b716-2dd951c3fd25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:58.284226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a5e68-1553-432b-b716-2dd951c3fd25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:58.287541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a5e68-1553-432b-b716-2dd951c3fd25 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:58.300909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 935/1866] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:17:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:17:59.463223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486a4ff-5ad7-48cc-a3f3-b72003d5c184 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:59.472928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486a4ff-5ad7-48cc-a3f3-b72003d5c184 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:59.486211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8c9856440cc4b36a07a647aa3ba087f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16cf2957975e4b6cbee19d683bf843d0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:59.486837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:59.487254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:59.487534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:59.488201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:59.488505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:59.490707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:59.503639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 932/1867] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:17:59.518012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5e68-1553-432b-b716-2dd951c3fd25 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:59.529541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5e68-1553-432b-b716-2dd951c3fd25 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:59.535856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664a1dd-5d50-498b-b181-f535db82d152 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8c9856440cc4b36a07a647aa3ba087f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16cf2957975e4b6cbee19d683bf843d0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:59.536497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664a1dd-5d50-498b-b181-f535db82d152 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:59.536900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664a1dd-5d50-498b-b181-f535db82d152 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:59.537043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664a1dd-5d50-498b-b181-f535db82d152 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:59.538646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664a1dd-5d50-498b-b181-f535db82d152 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:59.541168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664a1dd-5d50-498b-b181-f535db82d152 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:59.678516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 936/1868] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:17:59 2022] POST /identity/v3/projects => generated 353 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:17:59.680039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:17:59.690750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f8bdc-3b36-4c67-abce-d4ae2dfa0ee7 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:17:59.758933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8c9856440cc4b36a07a647aa3ba087f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16cf2957975e4b6cbee19d683bf843d0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:17:59.761770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:17:59.762263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:17:59.762931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:17:59.764004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:17:59.764526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:17:59.766981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:17:59.793661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 933/1869] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:18:00.112000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664a1dd-5d50-498b-b181-f535db82d152 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:00.125923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664a1dd-5d50-498b-b181-f535db82d152 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:00.132218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8c9856440cc4b36a07a647aa3ba087f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '16cf2957975e4b6cbee19d683bf843d0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:00.132991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:00.133280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:00.133559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] PATH_INFO: `/v3/projects/bb2482193cf04265ae6936e44ecdc522` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:00.134841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bb2482193cf04265ae6936e44ecdc522)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:00.137504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:00.329926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2482193cf04265ae6936e44ecdc522', 'request_id': 'req-03a45291-dcca-4748-8ac0-f04730e30c79'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:00.451722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2482193cf04265ae6936e44ecdc522', 'request_id': 'req-03a45291-dcca-4748-8ac0-f04730e30c79'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:00.460750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2482193cf04265ae6936e44ecdc522', 'request_id': 'req-03a45291-dcca-4748-8ac0-f04730e30c79'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:00.479160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] The token cache is being invalidate because project bb2482193cf04265ae6936e44ecdc522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:00.479512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:00.480286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:00.482217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 937/1870] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:00 2022] DELETE /identity/v3/projects/bb2482193cf04265ae6936e44ecdc522 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:01.294645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:01.296611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:01.352834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:01.354373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:01.370017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec991f-365c-41c0-9c56-e3335e511b2b tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:01.419413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:01.422630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a45291-dcca-4748-8ac0-f04730e30c79 tempest-VolumeQuotaClassesTest-1470762837 tempest-VolumeQuotaClassesTest-1470762837-project-admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:01.427508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:01.428142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:01.428429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:01.428698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] PATH_INFO: `/v3/users/ff073f4cdfca4fde8c76c71b616909c7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:01.429266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:01.429839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:01.430083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff073f4cdfca4fde8c76c71b616909c7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:01.430180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:01.430369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] PATH_INFO: `/v3/users/1625d2a762604d8e9b672d0258e069fb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:01.431527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1625d2a762604d8e9b672d0258e069fb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:01.432103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:01.433467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:01.434067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:01.980196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff073f4cdfca4fde8c76c71b616909c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:01.980368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1625d2a762604d8e9b672d0258e069fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:01.980912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1625d2a762604d8e9b672d0258e069fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:01.987138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff073f4cdfca4fde8c76c71b616909c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:01.987803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff073f4cdfca4fde8c76c71b616909c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:01.987941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1625d2a762604d8e9b672d0258e069fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:01.988411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff073f4cdfca4fde8c76c71b616909c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:01.988566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1625d2a762604d8e9b672d0258e069fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:02.001450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f052d902-43c2-414b-b35e-6328590c22c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff073f4cdfca4fde8c76c71b616909c7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:02.063564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-68e205ff-6473-4e36-a813-39b5f1223910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1625d2a762604d8e9b672d0258e069fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:02.081787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 934/1871] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:01 2022] DELETE /identity/v3/users/1625d2a762604d8e9b672d0258e069fb => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:02.086232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968a0ca-12b1-4f98-a3bc-af403ca00d18 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:02.086514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968a0ca-12b1-4f98-a3bc-af403ca00d18 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:02.086789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968a0ca-12b1-4f98-a3bc-af403ca00d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:02.105730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d968a0ca-12b1-4f98-a3bc-af403ca00d18 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:02.106963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d968a0ca-12b1-4f98-a3bc-af403ca00d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:02.120462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 938/1872] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:01 2022] DELETE /identity/v3/users/ff073f4cdfca4fde8c76c71b616909c7 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:02.124860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711773e1-af3a-4404-8173-66679d87e424 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:02.125150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711773e1-af3a-4404-8173-66679d87e424 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:02.125448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711773e1-af3a-4404-8173-66679d87e424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:02.144517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-711773e1-af3a-4404-8173-66679d87e424 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:02.145732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-711773e1-af3a-4404-8173-66679d87e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:02.166507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 935/1873] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:02.176610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968a0ca-12b1-4f98-a3bc-af403ca00d18 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:02.178574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d968a0ca-12b1-4f98-a3bc-af403ca00d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:02.207129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 939/1874] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:02.221901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711773e1-af3a-4404-8173-66679d87e424 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:02.223687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-711773e1-af3a-4404-8173-66679d87e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:02.238781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968a0ca-12b1-4f98-a3bc-af403ca00d18 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:02.267915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:02.293862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:02.294475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:02.295469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:02.296025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:02.299652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711773e1-af3a-4404-8173-66679d87e424 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:02.305596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:02.308073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:02.308073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:02.308073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:02.308335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:02.309273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:02.319303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:02.321071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:02.334959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 936/1875] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:02.336563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 940/1876] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:02.892896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:02.894482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:02.908198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b0a44-b85a-490d-9ae5-34d71c5c6099 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:02.913211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:02.913958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:02.914244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:02.914519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] PATH_INFO: `/v3/projects/b3caa0978fa741ef89acfb625fc5d66a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:02.915702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3caa0978fa741ef89acfb625fc5d66a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:02.917630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:02.921243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:02.922888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:02.973847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67e8dc-faa8-4758-91f3-af7a25b4f1c6 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:02.979745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:02.980487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:02.980764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:02.981037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] PATH_INFO: `/v3/projects/df2262b66c8f4020bc0d05cfb320bea1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:02.982180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2262b66c8f4020bc0d05cfb320bea1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:02.984146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:03.002097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:03.044046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3caa0978fa741ef89acfb625fc5d66a', 'request_id': 'req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.049601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3caa0978fa741ef89acfb625fc5d66a', 'request_id': 'req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.055859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3caa0978fa741ef89acfb625fc5d66a', 'request_id': 'req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.127796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2262b66c8f4020bc0d05cfb320bea1', 'request_id': 'req-46a3c13b-3d97-4b58-bc16-db381e1795cb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.151226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] The token cache is being invalidate because project b3caa0978fa741ef89acfb625fc5d66a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:03.151226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.151226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.151226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 937/1877] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:02 2022] DELETE /identity/v3/projects/b3caa0978fa741ef89acfb625fc5d66a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:03.151226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:03.152557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:03.219068 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8fc28-895e-4e98-ac47-c3c64a3fbb89 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:03.238087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2262b66c8f4020bc0d05cfb320bea1', 'request_id': 'req-46a3c13b-3d97-4b58-bc16-db381e1795cb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.245221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2262b66c8f4020bc0d05cfb320bea1', 'request_id': 'req-46a3c13b-3d97-4b58-bc16-db381e1795cb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.256916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] The token cache is being invalidate because project df2262b66c8f4020bc0d05cfb320bea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:03.257280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.257949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.259547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 941/1878] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:02 2022] DELETE /identity/v3/projects/df2262b66c8f4020bc0d05cfb320bea1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:03.263430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:03.264796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:03.276059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:03.276834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:03.277111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:03.277381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] PATH_INFO: `/v3/users/e8c9856440cc4b36a07a647aa3ba087f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:03.278550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c9856440cc4b36a07a647aa3ba087f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:03.280533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:03.282018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:03.328826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3c13b-3d97-4b58-bc16-db381e1795cb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:03.333720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:03.336809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:03.336809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:03.336809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] PATH_INFO: `/v3/users/030d61b39e7d43529c0a0a1478d64dda` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:03.336809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=030d61b39e7d43529c0a0a1478d64dda)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:03.338507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:03.494538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c9856440cc4b36a07a647aa3ba087f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.495337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c9856440cc4b36a07a647aa3ba087f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.502721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c9856440cc4b36a07a647aa3ba087f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.503434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c9856440cc4b36a07a647aa3ba087f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.657342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c9856440cc4b36a07a647aa3ba087f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.675809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 938/1879] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:03 2022] DELETE /identity/v3/users/e8c9856440cc4b36a07a647aa3ba087f => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:03.715181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030d61b39e7d43529c0a0a1478d64dda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.723524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030d61b39e7d43529c0a0a1478d64dda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.724106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030d61b39e7d43529c0a0a1478d64dda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.724726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030d61b39e7d43529c0a0a1478d64dda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.744226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030d61b39e7d43529c0a0a1478d64dda'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:03.829962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 942/1880] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:03 2022] DELETE /identity/v3/users/030d61b39e7d43529c0a0a1478d64dda => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:04.160581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:04.162351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:04.271259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65f030-8350-449f-af43-315bee652ea5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:04.332302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:04.332932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.333209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.333479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] PATH_INFO: `/v3/projects/16cf2957975e4b6cbee19d683bf843d0` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.335291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16cf2957975e4b6cbee19d683bf843d0)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:04.337723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:04.338288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:04.340345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:04.357787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd12f5-182d-482c-977b-9b4e07b645ba admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:04.363715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:04.364414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.364690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.364968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] PATH_INFO: `/v3/projects/8d7ede3fff2341d0941ae28aa572dcc9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.369191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7ede3fff2341d0941ae28aa572dcc9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:04.371569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:04.401298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:04.500391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:04.516247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16cf2957975e4b6cbee19d683bf843d0', 'request_id': 'req-4fed8b62-1387-4d68-9ac1-3231b7172e7d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.516751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7ede3fff2341d0941ae28aa572dcc9', 'request_id': 'req-73021b2b-03a6-494c-855f-6331edd7041e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.521663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16cf2957975e4b6cbee19d683bf843d0', 'request_id': 'req-4fed8b62-1387-4d68-9ac1-3231b7172e7d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.528213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16cf2957975e4b6cbee19d683bf843d0', 'request_id': 'req-4fed8b62-1387-4d68-9ac1-3231b7172e7d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.679113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7ede3fff2341d0941ae28aa572dcc9', 'request_id': 'req-73021b2b-03a6-494c-855f-6331edd7041e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.685776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7ede3fff2341d0941ae28aa572dcc9', 'request_id': 'req-73021b2b-03a6-494c-855f-6331edd7041e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.698588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] The token cache is being invalidate because project 8d7ede3fff2341d0941ae28aa572dcc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:04.698879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.699597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-73021b2b-03a6-494c-855f-6331edd7041e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.701090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 943/1881] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:04 2022] DELETE /identity/v3/projects/8d7ede3fff2341d0941ae28aa572dcc9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:04.706294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2c012-220e-47c3-bb2c-477e838fbade None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.706574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2c012-220e-47c3-bb2c-477e838fbade None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.706845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2c012-220e-47c3-bb2c-477e838fbade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.725898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a9e2c012-220e-47c3-bb2c-477e838fbade None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:04.727129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a9e2c012-220e-47c3-bb2c-477e838fbade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:04.754426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] The token cache is being invalidate because project 16cf2957975e4b6cbee19d683bf843d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:04.754712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.755562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4fed8b62-1387-4d68-9ac1-3231b7172e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:04.762981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 939/1882] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:04 2022] DELETE /identity/v3/projects/16cf2957975e4b6cbee19d683bf843d0 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:04.763367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c72f2-82e4-4769-bd51-23307cb5e938 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.763653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c72f2-82e4-4769-bd51-23307cb5e938 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.763933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c72f2-82e4-4769-bd51-23307cb5e938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.796976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-023c72f2-82e4-4769-bd51-23307cb5e938 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:04.798188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-023c72f2-82e4-4769-bd51-23307cb5e938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:04.817824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 944/1883] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:04.828657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 940/1884] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:04.832657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bab916-b83b-4167-8588-1aa96a185a4b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.832657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bab916-b83b-4167-8588-1aa96a185a4b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.832657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bab916-b83b-4167-8588-1aa96a185a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.832903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255997-eeab-4052-9fea-9c3265fe99b6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.832903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255997-eeab-4052-9fea-9c3265fe99b6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.833113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255997-eeab-4052-9fea-9c3265fe99b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.851315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-30bab916-b83b-4167-8588-1aa96a185a4b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:04.852540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-30bab916-b83b-4167-8588-1aa96a185a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:04.857702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c9255997-eeab-4052-9fea-9c3265fe99b6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:04.858880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c9255997-eeab-4052-9fea-9c3265fe99b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:04.861223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 945/1885] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:04.867295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bab916-b83b-4167-8588-1aa96a185a4b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:04.867806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 941/1886] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:04.871860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255997-eeab-4052-9fea-9c3265fe99b6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:04.876624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bab916-b83b-4167-8588-1aa96a185a4b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:04.881976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255997-eeab-4052-9fea-9c3265fe99b6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:04.882473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:04.883363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.883648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.883922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.884575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:04.884882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:04.886638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:04.886835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:04.887375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.887652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.887924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.888565 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:04.889972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:04.890545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:04.899044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 946/1887] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:18:04.906384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 942/1888] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:18:04.906514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:04.907563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:04.912988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239982-5b28-4a9b-a915-8183fc126cab admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:04.917292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507bf9b-5a56-4eb7-b881-62959f571f04 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:04.917588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cd9db-2251-49fb-bc1c-baf0bd0416a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:04.918173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cd9db-2251-49fb-bc1c-baf0bd0416a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.918454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cd9db-2251-49fb-bc1c-baf0bd0416a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.918729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cd9db-2251-49fb-bc1c-baf0bd0416a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.919444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cd9db-2251-49fb-bc1c-baf0bd0416a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:04.921805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cd9db-2251-49fb-bc1c-baf0bd0416a8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:04.922077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75aa0a-d11e-4023-b2a1-2d73007e50af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:04.922719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75aa0a-d11e-4023-b2a1-2d73007e50af admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:04.923029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75aa0a-d11e-4023-b2a1-2d73007e50af admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:04.923325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75aa0a-d11e-4023-b2a1-2d73007e50af admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:04.923929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75aa0a-d11e-4023-b2a1-2d73007e50af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:04.925982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75aa0a-d11e-4023-b2a1-2d73007e50af admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.041398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 947/1889] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:04 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:05.045535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cd9db-2251-49fb-bc1c-baf0bd0416a8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.053361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 943/1890] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:04 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:05.057340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75aa0a-d11e-4023-b2a1-2d73007e50af admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.111270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cd9db-2251-49fb-bc1c-baf0bd0416a8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.122861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:05.144367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75aa0a-d11e-4023-b2a1-2d73007e50af admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.150955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778259-19c1-4ef7-aeec-052476e4cb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.151587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778259-19c1-4ef7-aeec-052476e4cb69 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.151868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778259-19c1-4ef7-aeec-052476e4cb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.152139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778259-19c1-4ef7-aeec-052476e4cb69 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.152848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a778259-19c1-4ef7-aeec-052476e4cb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.154799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a778259-19c1-4ef7-aeec-052476e4cb69 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.175178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037b1df-85e3-430c-8cea-f12fe920cf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.175792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037b1df-85e3-430c-8cea-f12fe920cf21 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.176072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037b1df-85e3-430c-8cea-f12fe920cf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.176375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037b1df-85e3-430c-8cea-f12fe920cf21 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.177063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037b1df-85e3-430c-8cea-f12fe920cf21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.179083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037b1df-85e3-430c-8cea-f12fe920cf21 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.269279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 944/1891] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:05 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:05.273413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778259-19c1-4ef7-aeec-052476e4cb69 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.283553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778259-19c1-4ef7-aeec-052476e4cb69 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.288528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aedcc1-4a9b-42ee-9ad5-7ca81682db03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.289137 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aedcc1-4a9b-42ee-9ad5-7ca81682db03 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.289417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aedcc1-4a9b-42ee-9ad5-7ca81682db03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.289690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aedcc1-4a9b-42ee-9ad5-7ca81682db03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.290723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aedcc1-4a9b-42ee-9ad5-7ca81682db03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.292398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aedcc1-4a9b-42ee-9ad5-7ca81682db03 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.302393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 948/1892] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:05 2022] POST /identity/v3/users => generated 284 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:05.303828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037b1df-85e3-430c-8cea-f12fe920cf21 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.304304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 945/1893] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:05.309906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aedcc1-4a9b-42ee-9ad5-7ca81682db03 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.314820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037b1df-85e3-430c-8cea-f12fe920cf21 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.320384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4f1b3-a37d-4dfe-be7a-86923063f02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.320783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aedcc1-4a9b-42ee-9ad5-7ca81682db03 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.320995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4f1b3-a37d-4dfe-be7a-86923063f02f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.321279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4f1b3-a37d-4dfe-be7a-86923063f02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.321552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4f1b3-a37d-4dfe-be7a-86923063f02f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.322200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4f1b3-a37d-4dfe-be7a-86923063f02f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.323913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4f1b3-a37d-4dfe-be7a-86923063f02f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.325885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b8f2d-e13c-4e86-a6c2-450d0bc821e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.326487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b8f2d-e13c-4e86-a6c2-450d0bc821e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.326773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b8f2d-e13c-4e86-a6c2-450d0bc821e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.327073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b8f2d-e13c-4e86-a6c2-450d0bc821e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.327709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b8f2d-e13c-4e86-a6c2-450d0bc821e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.329573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b8f2d-e13c-4e86-a6c2-450d0bc821e8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.337049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 949/1894] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:05.339823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4f1b3-a37d-4dfe-be7a-86923063f02f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.346040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 946/1895] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:05.346987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b8f2d-e13c-4e86-a6c2-450d0bc821e8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.356849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b8f2d-e13c-4e86-a6c2-450d0bc821e8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.357745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4f1b3-a37d-4dfe-be7a-86923063f02f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.361447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb79a43-1d52-4fe0-9bf6-3ea47ebeed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.362076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79a43-1d52-4fe0-9bf6-3ea47ebeed11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.362349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79a43-1d52-4fe0-9bf6-3ea47ebeed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.363122 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb79a43-1d52-4fe0-9bf6-3ea47ebeed11 admin admin] PATH_INFO: `/v3/projects/0c32d78bc2324eb4be4f50683782ce6a/users/eb39df0476cd484d8879351b7dfe4822/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.363219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681c514-578b-4e17-a58d-dd9a886ad2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.363454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681c514-578b-4e17-a58d-dd9a886ad2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.363739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681c514-578b-4e17-a58d-dd9a886ad2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.364010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681c514-578b-4e17-a58d-dd9a886ad2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.364649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681c514-578b-4e17-a58d-dd9a886ad2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.367037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681c514-578b-4e17-a58d-dd9a886ad2a7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.378067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 950/1896] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:05.379041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb79a43-1d52-4fe0-9bf6-3ea47ebeed11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c32d78bc2324eb4be4f50683782ce6a, user_id=eb39df0476cd484d8879351b7dfe4822, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.382507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb79a43-1d52-4fe0-9bf6-3ea47ebeed11 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.383106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681c514-578b-4e17-a58d-dd9a886ad2a7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.393518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681c514-578b-4e17-a58d-dd9a886ad2a7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.398287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae88cdc-3256-471c-8efd-7aa150f2f5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.398881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88cdc-3256-471c-8efd-7aa150f2f5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.399194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88cdc-3256-471c-8efd-7aa150f2f5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.400130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88cdc-3256-471c-8efd-7aa150f2f5ad admin admin] PATH_INFO: `/v3/projects/92a5f50e8db24c9b8192e4e6d96990c4/users/ab32726bdd2d4607b191a0b422fbcd8c/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.415155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae88cdc-3256-471c-8efd-7aa150f2f5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a5f50e8db24c9b8192e4e6d96990c4, user_id=ab32726bdd2d4607b191a0b422fbcd8c, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.417827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae88cdc-3256-471c-8efd-7aa150f2f5ad admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.510992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 947/1897] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:05 2022] PUT /identity/v3/projects/0c32d78bc2324eb4be4f50683782ce6a/users/eb39df0476cd484d8879351b7dfe4822/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:05.521666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8c4d-2ae5-4f7b-a95a-79268a900841 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.522088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8c4d-2ae5-4f7b-a95a-79268a900841 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.522492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8c4d-2ae5-4f7b-a95a-79268a900841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.543962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 951/1898] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:05 2022] PUT /identity/v3/projects/92a5f50e8db24c9b8192e4e6d96990c4/users/ab32726bdd2d4607b191a0b422fbcd8c/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:05.552407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09409-4488-438a-b19b-dd0c087b5b2a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.552923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09409-4488-438a-b19b-dd0c087b5b2a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.553955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a09409-4488-438a-b19b-dd0c087b5b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.569527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:05.578514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3ebf8c4d-2ae5-4f7b-a95a-79268a900841 None None] MFA Rules not processed for user `eb39df0476cd484d8879351b7dfe4822`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:05.581112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3ebf8c4d-2ae5-4f7b-a95a-79268a900841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:05.592228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b7a09409-4488-438a-b19b-dd0c087b5b2a None None] MFA Rules not processed for user `ab32726bdd2d4607b191a0b422fbcd8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:05.593508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b7a09409-4488-438a-b19b-dd0c087b5b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:05.658385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 948/1899] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:05.663770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8c4d-2ae5-4f7b-a95a-79268a900841 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.674967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 952/1900] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:05.680336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a09409-4488-438a-b19b-dd0c087b5b2a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.741926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:05.747283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8c4d-2ae5-4f7b-a95a-79268a900841 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.752510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d121a-3888-4970-99c9-d759fdf05274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.753123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d121a-3888-4970-99c9-d759fdf05274 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.753402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d121a-3888-4970-99c9-d759fdf05274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.753673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d121a-3888-4970-99c9-d759fdf05274 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.754295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d121a-3888-4970-99c9-d759fdf05274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.756302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d121a-3888-4970-99c9-d759fdf05274 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.758088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a09409-4488-438a-b19b-dd0c087b5b2a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.762898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ea514-1d75-4c92-ab2f-ad0e9d5c0840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.763529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea514-1d75-4c92-ab2f-ad0e9d5c0840 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.763805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea514-1d75-4c92-ab2f-ad0e9d5c0840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.764075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea514-1d75-4c92-ab2f-ad0e9d5c0840 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.764756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ea514-1d75-4c92-ab2f-ad0e9d5c0840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.766651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ea514-1d75-4c92-ab2f-ad0e9d5c0840 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.843417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 949/1901] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:05 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:05.847115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d121a-3888-4970-99c9-d759fdf05274 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.887924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 953/1902] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:05 2022] POST /identity/v3/projects => generated 367 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:05.891743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ea514-1d75-4c92-ab2f-ad0e9d5c0840 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:05.930978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d121a-3888-4970-99c9-d759fdf05274 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.974584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ea514-1d75-4c92-ab2f-ad0e9d5c0840 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:05.980366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1fe34-c8a2-4b96-9078-2f3e70ddfef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:05.980975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1fe34-c8a2-4b96-9078-2f3e70ddfef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:05.981253 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1fe34-c8a2-4b96-9078-2f3e70ddfef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:05.981556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1fe34-c8a2-4b96-9078-2f3e70ddfef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:05.982182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1fe34-c8a2-4b96-9078-2f3e70ddfef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:05.984196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1fe34-c8a2-4b96-9078-2f3e70ddfef0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:05.985511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:06.014620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366aec0-2594-49b4-a71f-c7d6124490b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.015338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366aec0-2594-49b4-a71f-c7d6124490b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.015673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366aec0-2594-49b4-a71f-c7d6124490b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.016004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366aec0-2594-49b4-a71f-c7d6124490b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.016802 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366aec0-2594-49b4-a71f-c7d6124490b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.018786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366aec0-2594-49b4-a71f-c7d6124490b7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.137646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 950/1903] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:05 2022] POST /identity/v3/users => generated 289 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:06.141316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 954/1904] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:05 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:06.141480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366aec0-2594-49b4-a71f-c7d6124490b7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.142488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1fe34-c8a2-4b96-9078-2f3e70ddfef0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.152937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1fe34-c8a2-4b96-9078-2f3e70ddfef0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.157330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366aec0-2594-49b4-a71f-c7d6124490b7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.157852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c79b9-7822-423b-8bad-54c83c2e895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.157976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79810a1-9e7d-4271-abf6-d65548d0daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.158440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79810a1-9e7d-4271-abf6-d65548d0daab admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.158639 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c79b9-7822-423b-8bad-54c83c2e895c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.158733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79810a1-9e7d-4271-abf6-d65548d0daab admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.159045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79810a1-9e7d-4271-abf6-d65548d0daab admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.159139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c79b9-7822-423b-8bad-54c83c2e895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.159452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c79b9-7822-423b-8bad-54c83c2e895c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.159718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79810a1-9e7d-4271-abf6-d65548d0daab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.160220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c79b9-7822-423b-8bad-54c83c2e895c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.161361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79810a1-9e7d-4271-abf6-d65548d0daab admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.162050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c79b9-7822-423b-8bad-54c83c2e895c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.173872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 951/1905] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:06.180264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 955/1906] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:06.181176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c79b9-7822-423b-8bad-54c83c2e895c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.181291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79810a1-9e7d-4271-abf6-d65548d0daab admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.192144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79810a1-9e7d-4271-abf6-d65548d0daab admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.192346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c79b9-7822-423b-8bad-54c83c2e895c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.196809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e79fce-0b0f-481d-b9e2-908429f72381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.197105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce4fbc-5433-48f3-84ab-03647044f7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.197480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e79fce-0b0f-481d-b9e2-908429f72381 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.197745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce4fbc-5433-48f3-84ab-03647044f7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.197864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e79fce-0b0f-481d-b9e2-908429f72381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.198032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce4fbc-5433-48f3-84ab-03647044f7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.198127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e79fce-0b0f-481d-b9e2-908429f72381 admin admin] PATH_INFO: `/v3/projects/b2b1d41923af4869b4ab8875f1ff7d36/users/a023ca9fcca14b03a150a786ca898d24/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.198292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce4fbc-5433-48f3-84ab-03647044f7d1 admin admin] PATH_INFO: `/v3/projects/3eaa6e02f70d40eaa84f1bdeb2cd95d9/users/9a8c403b206844ffa42fdcf97050795a/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.214055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e79fce-0b0f-481d-b9e2-908429f72381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b1d41923af4869b4ab8875f1ff7d36, user_id=a023ca9fcca14b03a150a786ca898d24, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.215441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce4fbc-5433-48f3-84ab-03647044f7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eaa6e02f70d40eaa84f1bdeb2cd95d9, user_id=9a8c403b206844ffa42fdcf97050795a, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.216885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e79fce-0b0f-481d-b9e2-908429f72381 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.218311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce4fbc-5433-48f3-84ab-03647044f7d1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.297355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 956/1907] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:06 2022] PUT /identity/v3/projects/b2b1d41923af4869b4ab8875f1ff7d36/users/a023ca9fcca14b03a150a786ca898d24/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:06.301178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e79fce-0b0f-481d-b9e2-908429f72381 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.367192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 952/1908] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:06 2022] PUT /identity/v3/projects/3eaa6e02f70d40eaa84f1bdeb2cd95d9/users/9a8c403b206844ffa42fdcf97050795a/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:06.367340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e79fce-0b0f-481d-b9e2-908429f72381 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.368406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce4fbc-5433-48f3-84ab-03647044f7d1 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.404796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:06.444414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce4fbc-5433-48f3-84ab-03647044f7d1 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.446455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8592bb-0489-4894-8d8a-36340932d73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.447109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8592bb-0489-4894-8d8a-36340932d73c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.447446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8592bb-0489-4894-8d8a-36340932d73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.447720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8592bb-0489-4894-8d8a-36340932d73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.448389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8592bb-0489-4894-8d8a-36340932d73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.449228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b1df-3914-43fb-a6bd-2dbd03fa82f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.449970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8592bb-0489-4894-8d8a-36340932d73c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.450077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b1df-3914-43fb-a6bd-2dbd03fa82f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.450387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b1df-3914-43fb-a6bd-2dbd03fa82f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.450719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b1df-3914-43fb-a6bd-2dbd03fa82f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.451530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407b1df-3914-43fb-a6bd-2dbd03fa82f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.453194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407b1df-3914-43fb-a6bd-2dbd03fa82f3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.461210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 957/1909] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:06.465504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8592bb-0489-4894-8d8a-36340932d73c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.471794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 953/1910] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:06.477234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8592bb-0489-4894-8d8a-36340932d73c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.480110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b1df-3914-43fb-a6bd-2dbd03fa82f3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.482675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3df8d-3ec9-4ac0-87c5-14c6d4745044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.483441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3df8d-3ec9-4ac0-87c5-14c6d4745044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.483721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3df8d-3ec9-4ac0-87c5-14c6d4745044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.483991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3df8d-3ec9-4ac0-87c5-14c6d4745044 admin admin] PATH_INFO: `/v3/projects/b2b1d41923af4869b4ab8875f1ff7d36/users/a023ca9fcca14b03a150a786ca898d24/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.486272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3df8d-3ec9-4ac0-87c5-14c6d4745044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b1d41923af4869b4ab8875f1ff7d36, user_id=a023ca9fcca14b03a150a786ca898d24, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.489717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3df8d-3ec9-4ac0-87c5-14c6d4745044 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.491733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b1df-3914-43fb-a6bd-2dbd03fa82f3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.496993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbf3fa-cb86-4656-b2e4-5e2ebfaf442e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.497651 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbf3fa-cb86-4656-b2e4-5e2ebfaf442e admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.497936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbf3fa-cb86-4656-b2e4-5e2ebfaf442e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.498235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbf3fa-cb86-4656-b2e4-5e2ebfaf442e admin admin] PATH_INFO: `/v3/projects/3eaa6e02f70d40eaa84f1bdeb2cd95d9/users/9a8c403b206844ffa42fdcf97050795a/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.500080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 958/1911] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:06 2022] PUT /identity/v3/projects/b2b1d41923af4869b4ab8875f1ff7d36/users/a023ca9fcca14b03a150a786ca898d24/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:06.500207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbf3fa-cb86-4656-b2e4-5e2ebfaf442e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eaa6e02f70d40eaa84f1bdeb2cd95d9, user_id=9a8c403b206844ffa42fdcf97050795a, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.502987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbf3fa-cb86-4656-b2e4-5e2ebfaf442e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.514402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 954/1912] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:06 2022] PUT /identity/v3/projects/3eaa6e02f70d40eaa84f1bdeb2cd95d9/users/9a8c403b206844ffa42fdcf97050795a/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:06.515191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af477-065d-4229-ab85-8282911b12ce None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.515480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af477-065d-4229-ab85-8282911b12ce None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.515780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af477-065d-4229-ab85-8282911b12ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.525723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c8a9-923e-4a22-b4ee-6a58ae83a693 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.526007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c8a9-923e-4a22-b4ee-6a58ae83a693 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.526282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c8a9-923e-4a22-b4ee-6a58ae83a693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.544344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d70af477-065d-4229-ab85-8282911b12ce None None] MFA Rules not processed for user `a023ca9fcca14b03a150a786ca898d24`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:06.545561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d70af477-065d-4229-ab85-8282911b12ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:06.556789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-25f4c8a9-923e-4a22-b4ee-6a58ae83a693 None None] MFA Rules not processed for user `9a8c403b206844ffa42fdcf97050795a`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:06.557710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-25f4c8a9-923e-4a22-b4ee-6a58ae83a693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:06.639585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 959/1913] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:06.646283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:06.654080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70af477-065d-4229-ab85-8282911b12ce None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.655683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d70af477-065d-4229-ab85-8282911b12ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:06.657281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 955/1914] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:06.673049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4c8a9-923e-4a22-b4ee-6a58ae83a693 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:06.674794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-25f4c8a9-923e-4a22-b4ee-6a58ae83a693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:06.726450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70af477-065d-4229-ab85-8282911b12ce None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.730235 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981190bb-94f8-4eec-a622-2a652b556a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.730875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981190bb-94f8-4eec-a622-2a652b556a3c service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.731199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981190bb-94f8-4eec-a622-2a652b556a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.731473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981190bb-94f8-4eec-a622-2a652b556a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.734397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4c8a9-923e-4a22-b4ee-6a58ae83a693 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:06.738226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:06.738909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:06.739118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:06.739386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:06.740809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981190bb-94f8-4eec-a622-2a652b556a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.742310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981190bb-94f8-4eec-a622-2a652b556a3c service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.753607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:06.753778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 960/1915] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:06.764151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:06.783463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 956/1916] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:07.256106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981190bb-94f8-4eec-a622-2a652b556a3c service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:07.265257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981190bb-94f8-4eec-a622-2a652b556a3c service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:07.268956 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0a6d-8cc9-4915-9138-ebddb6b62ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:07.269555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0a6d-8cc9-4915-9138-ebddb6b62ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:07.269843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0a6d-8cc9-4915-9138-ebddb6b62ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:07.270147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0a6d-8cc9-4915-9138-ebddb6b62ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:07.322806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:07.339055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0a6d-8cc9-4915-9138-ebddb6b62ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:07.340526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0a6d-8cc9-4915-9138-ebddb6b62ce2 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:07.350517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 961/1917] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:08.905237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:08.906671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:08.977128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bb006-fc8c-4f8c-9e1c-f349e57c00ab service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:08.981829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:08.982450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:08.982729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:08.983036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] PATH_INFO: `/v3/users/13c1b99d6b5b4847a1c3c1aff74facc0` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:08.984151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c1b99d6b5b4847a1c3c1aff74facc0)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:08.986086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:09.239636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c1b99d6b5b4847a1c3c1aff74facc0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:09.240293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c1b99d6b5b4847a1c3c1aff74facc0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:09.248221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c1b99d6b5b4847a1c3c1aff74facc0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:09.249184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c1b99d6b5b4847a1c3c1aff74facc0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:09.379099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c1b99d6b5b4847a1c3c1aff74facc0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:09.400671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 957/1918] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:08 2022] DELETE /identity/v3/users/13c1b99d6b5b4847a1c3c1aff74facc0 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:09.405454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05a640-2e0a-4864-90b7-1fe36fb25117 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:09.405908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05a640-2e0a-4864-90b7-1fe36fb25117 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:09.405999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05a640-2e0a-4864-90b7-1fe36fb25117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:09.427778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ad05a640-2e0a-4864-90b7-1fe36fb25117 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:09.429668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ad05a640-2e0a-4864-90b7-1fe36fb25117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:09.501324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 962/1919] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:09.523934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:09.525392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:09.547952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:09.590318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aa49f-659b-41e0-8335-db2cabe2e6c2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:09.594079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49388ab0-e4a3-4c97-b6e9-689664768354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:09.594697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388ab0-e4a3-4c97-b6e9-689664768354 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:09.594996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388ab0-e4a3-4c97-b6e9-689664768354 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:09.595272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388ab0-e4a3-4c97-b6e9-689664768354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:09.606498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49388ab0-e4a3-4c97-b6e9-689664768354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:09.608006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49388ab0-e4a3-4c97-b6e9-689664768354 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:09.618522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 958/1920] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:10.110004 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05a640-2e0a-4864-90b7-1fe36fb25117 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:10.111694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ad05a640-2e0a-4864-90b7-1fe36fb25117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:10.124435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05a640-2e0a-4864-90b7-1fe36fb25117 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:10.129144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:10.129781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:10.130059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:10.130332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] PATH_INFO: `/v3/projects/dd4db37a4b194e129570876b1e277dfd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:10.131998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4db37a4b194e129570876b1e277dfd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:10.133950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:10.300340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4db37a4b194e129570876b1e277dfd', 'request_id': 'req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:10.372922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4db37a4b194e129570876b1e277dfd', 'request_id': 'req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:10.383450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4db37a4b194e129570876b1e277dfd', 'request_id': 'req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:10.403198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] The token cache is being invalidate because project dd4db37a4b194e129570876b1e277dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:10.403853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:10.405476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1ad04fef-4ec2-490d-abb7-13a6c41d57e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:10.407947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 963/1921] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:10 2022] DELETE /identity/v3/projects/dd4db37a4b194e129570876b1e277dfd => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:10.417939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e072-14ef-4dc0-88b1-c5a0c6adbad0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:10.417939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e072-14ef-4dc0-88b1-c5a0c6adbad0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:10.417939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e072-14ef-4dc0-88b1-c5a0c6adbad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:10.442573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-77c4e072-14ef-4dc0-88b1-c5a0c6adbad0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:10.443855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-77c4e072-14ef-4dc0-88b1-c5a0c6adbad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:10.527360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 959/1922] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:10.534554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81746-c085-40fe-ac05-05bdabb6c1d8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:10.534845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81746-c085-40fe-ac05-05bdabb6c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:10.535164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81746-c085-40fe-ac05-05bdabb6c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:10.563984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-97d81746-c085-40fe-ac05-05bdabb6c1d8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:10.565210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-97d81746-c085-40fe-ac05-05bdabb6c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:10.576080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 964/1923] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:10.580576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e072-14ef-4dc0-88b1-c5a0c6adbad0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:10.591634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e072-14ef-4dc0-88b1-c5a0c6adbad0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:10.596538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:10.597156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:10.597439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:10.597713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:10.598372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:10.598795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:10.600595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:10.613009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 960/1924] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:18:10.617925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d81746-c085-40fe-ac05-05bdabb6c1d8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:10.628050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d81746-c085-40fe-ac05-05bdabb6c1d8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:10.633081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e21d1-0c24-4a51-ad6f-85a9c25e23b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:10.633748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e21d1-0c24-4a51-ad6f-85a9c25e23b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:10.634034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e21d1-0c24-4a51-ad6f-85a9c25e23b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:10.641328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e21d1-0c24-4a51-ad6f-85a9c25e23b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:10.641328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e21d1-0c24-4a51-ad6f-85a9c25e23b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:10.641328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e21d1-0c24-4a51-ad6f-85a9c25e23b1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:10.761408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 965/1925] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:10 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:10.765872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:10.778646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4453ff6-9c33-4876-a1e3-b3b0e47309fa admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:10.868976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfe643-7bd5-4c07-a8b2-418304087bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:10.869620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfe643-7bd5-4c07-a8b2-418304087bee admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:10.869941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfe643-7bd5-4c07-a8b2-418304087bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:10.870376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfe643-7bd5-4c07-a8b2-418304087bee admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:10.871077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfe643-7bd5-4c07-a8b2-418304087bee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:10.873370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfe643-7bd5-4c07-a8b2-418304087bee admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.029762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 961/1926] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:10 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:11.038907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e21d1-0c24-4a51-ad6f-85a9c25e23b1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.050695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e21d1-0c24-4a51-ad6f-85a9c25e23b1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.056018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026f36-0d31-40d0-b3a8-c7db3d19b3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.056677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026f36-0d31-40d0-b3a8-c7db3d19b3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.056988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026f36-0d31-40d0-b3a8-c7db3d19b3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.057264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026f36-0d31-40d0-b3a8-c7db3d19b3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.057964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9026f36-0d31-40d0-b3a8-c7db3d19b3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:11.059631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9026f36-0d31-40d0-b3a8-c7db3d19b3b1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.070324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 966/1927] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:11.074538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfe643-7bd5-4c07-a8b2-418304087bee admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.085297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfe643-7bd5-4c07-a8b2-418304087bee admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.090001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04057e8d-b2cb-45fa-bac4-70b80da14749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.090591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057e8d-b2cb-45fa-bac4-70b80da14749 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.090868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057e8d-b2cb-45fa-bac4-70b80da14749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.091306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057e8d-b2cb-45fa-bac4-70b80da14749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.091853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04057e8d-b2cb-45fa-bac4-70b80da14749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:11.093440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04057e8d-b2cb-45fa-bac4-70b80da14749 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.105697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 962/1928] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:11.109986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026f36-0d31-40d0-b3a8-c7db3d19b3b1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.121114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026f36-0d31-40d0-b3a8-c7db3d19b3b1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.126206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7421e-60c3-446b-85fa-206bc0e5a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.126861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7421e-60c3-446b-85fa-206bc0e5a336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.127282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7421e-60c3-446b-85fa-206bc0e5a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.127565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7421e-60c3-446b-85fa-206bc0e5a336 admin admin] PATH_INFO: `/v3/projects/44fa1023082c41ca9cd47378cbf3c191/users/e3449215c6874ea69749ec369cc508b9/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.143235 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7421e-60c3-446b-85fa-206bc0e5a336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44fa1023082c41ca9cd47378cbf3c191, user_id=e3449215c6874ea69749ec369cc508b9, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:11.145740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7421e-60c3-446b-85fa-206bc0e5a336 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.224732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 967/1929] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:11 2022] PUT /identity/v3/projects/44fa1023082c41ca9cd47378cbf3c191/users/e3449215c6874ea69749ec369cc508b9/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:11.235805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea960d9a-fe12-4417-a849-070236daa4c4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.236103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea960d9a-fe12-4417-a849-070236daa4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.236358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea960d9a-fe12-4417-a849-070236daa4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.267892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ea960d9a-fe12-4417-a849-070236daa4c4 None None] MFA Rules not processed for user `e3449215c6874ea69749ec369cc508b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:11.269300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ea960d9a-fe12-4417-a849-070236daa4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:11.289741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:11.379090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 963/1930] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:11.382283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7421e-60c3-446b-85fa-206bc0e5a336 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.390985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:11.461106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7421e-60c3-446b-85fa-206bc0e5a336 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.466848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb995d-b089-4adc-ae68-a3c84d83e6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.467579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb995d-b089-4adc-ae68-a3c84d83e6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.467960 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb995d-b089-4adc-ae68-a3c84d83e6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.468378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb995d-b089-4adc-ae68-a3c84d83e6e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.469149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb995d-b089-4adc-ae68-a3c84d83e6e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:11.471179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb995d-b089-4adc-ae68-a3c84d83e6e1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.617649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 968/1931] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:11 2022] POST /identity/v3/projects => generated 377 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:11.618535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea960d9a-fe12-4417-a849-070236daa4c4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.630059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea960d9a-fe12-4417-a849-070236daa4c4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.691440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.692045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.692324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.692596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.693269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:11.695250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.824660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 964/1932] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:11 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:11.828532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb995d-b089-4adc-ae68-a3c84d83e6e1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.848471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.856223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb995d-b089-4adc-ae68-a3c84d83e6e1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.857539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:11.874774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8e4df-8b73-48b0-b957-cec2849fc504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.875352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8e4df-8b73-48b0-b957-cec2849fc504 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.875626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8e4df-8b73-48b0-b957-cec2849fc504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.875893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8e4df-8b73-48b0-b957-cec2849fc504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.876719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8e4df-8b73-48b0-b957-cec2849fc504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:11.884778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8e4df-8b73-48b0-b957-cec2849fc504 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.890890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1574-7090-4241-9660-b480f8aaf45a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.908262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 969/1933] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:11.912335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.913933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.914738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.915047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] PATH_INFO: `/v3/users/4e7b70887d8d4bbc8c30a2ba260bdf8f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.916113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8e4df-8b73-48b0-b957-cec2849fc504 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.916247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e7b70887d8d4bbc8c30a2ba260bdf8f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:11.918278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.927459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8e4df-8b73-48b0-b957-cec2849fc504 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.933530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f8557-5942-4f03-9d2a-fd0e29a0c568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.939573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8557-5942-4f03-9d2a-fd0e29a0c568 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.939573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8557-5942-4f03-9d2a-fd0e29a0c568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.939573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8557-5942-4f03-9d2a-fd0e29a0c568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:11.939573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f8557-5942-4f03-9d2a-fd0e29a0c568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:11.939573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f8557-5942-4f03-9d2a-fd0e29a0c568 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:11.948366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 970/1934] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:11.952388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f8557-5942-4f03-9d2a-fd0e29a0c568 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:11.965778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f8557-5942-4f03-9d2a-fd0e29a0c568 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:11.971943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e72c96-aa01-4f30-9ee5-27e32cbb0f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:11.972311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72c96-aa01-4f30-9ee5-27e32cbb0f64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:11.974119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72c96-aa01-4f30-9ee5-27e32cbb0f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:11.974119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e72c96-aa01-4f30-9ee5-27e32cbb0f64 admin admin] PATH_INFO: `/v3/projects/fe9bac65b3a0465db5336b698d88bebe/users/e9e4c649ec5d418486aec2cd44ac4c41/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:12.001152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e72c96-aa01-4f30-9ee5-27e32cbb0f64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9bac65b3a0465db5336b698d88bebe, user_id=e9e4c649ec5d418486aec2cd44ac4c41, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:12.004305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e72c96-aa01-4f30-9ee5-27e32cbb0f64 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:12.108314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 971/1935] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:11 2022] PUT /identity/v3/projects/fe9bac65b3a0465db5336b698d88bebe/users/e9e4c649ec5d418486aec2cd44ac4c41/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:12.119084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000b8c-0a48-4d77-b02b-aa779ae813b0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:12.119371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000b8c-0a48-4d77-b02b-aa779ae813b0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:12.119678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000b8c-0a48-4d77-b02b-aa779ae813b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:12.152678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e7b70887d8d4bbc8c30a2ba260bdf8f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.153922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e7b70887d8d4bbc8c30a2ba260bdf8f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.181483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e7b70887d8d4bbc8c30a2ba260bdf8f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.181483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e7b70887d8d4bbc8c30a2ba260bdf8f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.191886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4f000b8c-0a48-4d77-b02b-aa779ae813b0 None None] MFA Rules not processed for user `e9e4c649ec5d418486aec2cd44ac4c41`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:12.194846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4f000b8c-0a48-4d77-b02b-aa779ae813b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:12.275917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f8e713d6-7512-4f88-bb33-bc829d1b112d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e7b70887d8d4bbc8c30a2ba260bdf8f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.281057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 972/1936] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:12.285224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000b8c-0a48-4d77-b02b-aa779ae813b0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:12.286903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4f000b8c-0a48-4d77-b02b-aa779ae813b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:12.297745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 965/1937] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:11 2022] DELETE /identity/v3/users/4e7b70887d8d4bbc8c30a2ba260bdf8f => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:12.302211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390557b-da06-4b63-8d5c-adccc8eef65a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:12.302873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390557b-da06-4b63-8d5c-adccc8eef65a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:12.304542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390557b-da06-4b63-8d5c-adccc8eef65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:12.328544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5390557b-da06-4b63-8d5c-adccc8eef65a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:12.329769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5390557b-da06-4b63-8d5c-adccc8eef65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:12.329883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:12.345130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:12.371909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000b8c-0a48-4d77-b02b-aa779ae813b0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:12.438575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 966/1938] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:12.442888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390557b-da06-4b63-8d5c-adccc8eef65a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:12.445046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5390557b-da06-4b63-8d5c-adccc8eef65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:12.457869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:12.458684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:12.459080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:12.459406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:12.459909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390557b-da06-4b63-8d5c-adccc8eef65a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:12.463872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:12.463872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:12.465695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:12.466335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:12.466614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:12.466888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] PATH_INFO: `/v3/users/ab32726bdd2d4607b191a0b422fbcd8c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:12.468286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab32726bdd2d4607b191a0b422fbcd8c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:12.470295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:12.673361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 973/1939] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:12 2022] POST /identity/v3/projects => generated 375 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:12.674463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:12.685630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:12.747068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e9e60-8750-49ab-bd14-073de27d3d28 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:12.750714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35922a85-79fb-4fa5-b164-8b376815243b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:12.751348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35922a85-79fb-4fa5-b164-8b376815243b service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:12.751626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35922a85-79fb-4fa5-b164-8b376815243b service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:12.752032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35922a85-79fb-4fa5-b164-8b376815243b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:12.771190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab32726bdd2d4607b191a0b422fbcd8c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.771845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab32726bdd2d4607b191a0b422fbcd8c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.779036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab32726bdd2d4607b191a0b422fbcd8c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.779705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab32726bdd2d4607b191a0b422fbcd8c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.823670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35922a85-79fb-4fa5-b164-8b376815243b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:12.825121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35922a85-79fb-4fa5-b164-8b376815243b service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:12.835415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 974/1940] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:12.836148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35922a85-79fb-4fa5-b164-8b376815243b service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:12.845822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35922a85-79fb-4fa5-b164-8b376815243b service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:12.851808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:12.852576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:12.852858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:12.853127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:12.853752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:12.855765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:12.876240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:12.892658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e3d1939a-a528-4247-9ebf-dc9ad75d4cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab32726bdd2d4607b191a0b422fbcd8c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:12.915047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 967/1941] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:12 2022] DELETE /identity/v3/users/ab32726bdd2d4607b191a0b422fbcd8c => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:12.919342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f660-1132-4532-affb-994140146a64 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:12.919870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f660-1132-4532-affb-994140146a64 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:12.920232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f660-1132-4532-affb-994140146a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:12.970953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-11d5f660-1132-4532-affb-994140146a64 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:12.973169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-11d5f660-1132-4532-affb-994140146a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:12.993250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 975/1942] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:12 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:13.002511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:13.003033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:13.037157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca29a25-26eb-4d76-81f2-da7e58cd8a0d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:13.118235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 968/1943] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:13.136301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5f660-1132-4532-affb-994140146a64 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:13.141824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-11d5f660-1132-4532-affb-994140146a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:13.180028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696c26b-c7d2-4704-a7d0-0d9426c868dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:13.180867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696c26b-c7d2-4704-a7d0-0d9426c868dd admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:13.181291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696c26b-c7d2-4704-a7d0-0d9426c868dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:13.185024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696c26b-c7d2-4704-a7d0-0d9426c868dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:13.186722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696c26b-c7d2-4704-a7d0-0d9426c868dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:13.189985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696c26b-c7d2-4704-a7d0-0d9426c868dd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:13.218370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 976/1944] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:13.219484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696c26b-c7d2-4704-a7d0-0d9426c868dd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:13.257424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696c26b-c7d2-4704-a7d0-0d9426c868dd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:13.273717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:13.277311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:13.278019 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:13.279082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] PATH_INFO: `/v3/projects/4bd2f1ea35164d4399baae86a56bb45e/users/ba76f05db5094413989c32dbc323345c/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:13.301314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd2f1ea35164d4399baae86a56bb45e, user_id=ba76f05db5094413989c32dbc323345c, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:13.302507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:13.305095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:13.364663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5f660-1132-4532-affb-994140146a64 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:13.369223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6415c0-8a32-4e5b-a703-9540ba3e370b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:13.369912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6415c0-8a32-4e5b-a703-9540ba3e370b service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:13.370265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6415c0-8a32-4e5b-a703-9540ba3e370b service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:13.370607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6415c0-8a32-4e5b-a703-9540ba3e370b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:13.381656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6415c0-8a32-4e5b-a703-9540ba3e370b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:13.383286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6415c0-8a32-4e5b-a703-9540ba3e370b service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:13.394612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 969/1945] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:13.483883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 977/1946] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:13 2022] PUT /identity/v3/projects/4bd2f1ea35164d4399baae86a56bb45e/users/ba76f05db5094413989c32dbc323345c/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:13.487978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6415c0-8a32-4e5b-a703-9540ba3e370b service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:13.502588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6415c0-8a32-4e5b-a703-9540ba3e370b service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:13.550274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:13.552362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:13.657119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbde692-4f15-454d-a65b-e56dc9103cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:13.657810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbde692-4f15-454d-a65b-e56dc9103cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:13.658229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbde692-4f15-454d-a65b-e56dc9103cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:13.658505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbde692-4f15-454d-a65b-e56dc9103cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:13.666731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbde692-4f15-454d-a65b-e56dc9103cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:13.666731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbde692-4f15-454d-a65b-e56dc9103cfe admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:13.679975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3037c4-ff78-4022-800b-4c708e55ba09 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:13.690352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:13.694365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:13.695399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:13.695687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] PATH_INFO: `/v3/projects/c8fa2a8460b041c4a72e57c2ef35bc8d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:13.699605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 970/1947] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:13.699730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8fa2a8460b041c4a72e57c2ef35bc8d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:13.699730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:13.703062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbde692-4f15-454d-a65b-e56dc9103cfe admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:13.714201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbde692-4f15-454d-a65b-e56dc9103cfe admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:13.720767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cfa36-7ba2-40c4-9a92-5cbdfea9afe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:13.721069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cfa36-7ba2-40c4-9a92-5cbdfea9afe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:13.721596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cfa36-7ba2-40c4-9a92-5cbdfea9afe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:13.721943 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cfa36-7ba2-40c4-9a92-5cbdfea9afe4 admin admin] PATH_INFO: `/v3/projects/4bd2f1ea35164d4399baae86a56bb45e/users/ba76f05db5094413989c32dbc323345c/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:13.724475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cfa36-7ba2-40c4-9a92-5cbdfea9afe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd2f1ea35164d4399baae86a56bb45e, user_id=ba76f05db5094413989c32dbc323345c, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:13.727575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cfa36-7ba2-40c4-9a92-5cbdfea9afe4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:13.740308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 971/1948] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:13 2022] PUT /identity/v3/projects/4bd2f1ea35164d4399baae86a56bb45e/users/ba76f05db5094413989c32dbc323345c/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:13.752199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c30d4-8ffd-4b91-b843-ca153b14a80e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:13.752486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c30d4-8ffd-4b91-b843-ca153b14a80e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:13.752764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c30d4-8ffd-4b91-b843-ca153b14a80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:13.791760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-019c30d4-8ffd-4b91-b843-ca153b14a80e None None] MFA Rules not processed for user `ba76f05db5094413989c32dbc323345c`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:13.793021 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-019c30d4-8ffd-4b91-b843-ca153b14a80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:13.906322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8fa2a8460b041c4a72e57c2ef35bc8d', 'request_id': 'req-fdcf1a71-52ea-4382-b456-179bc9977550'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:13.908803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 972/1949] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:14.036231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8fa2a8460b041c4a72e57c2ef35bc8d', 'request_id': 'req-fdcf1a71-52ea-4382-b456-179bc9977550'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.042934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8fa2a8460b041c4a72e57c2ef35bc8d', 'request_id': 'req-fdcf1a71-52ea-4382-b456-179bc9977550'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.051785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:14.057284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] The token cache is being invalidate because project c8fa2a8460b041c4a72e57c2ef35bc8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:14.057284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.058235 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.060173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 978/1950] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:13 2022] DELETE /identity/v3/projects/c8fa2a8460b041c4a72e57c2ef35bc8d => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:14.064604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c30d4-8ffd-4b91-b843-ca153b14a80e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:14.130654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c30d4-8ffd-4b91-b843-ca153b14a80e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:14.205272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:14.206734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:14.207229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:14.207655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] PATH_INFO: `/v3/users/95673ebd60e94d5faa6e46866d94fbba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:14.209696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95673ebd60e94d5faa6e46866d94fbba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:14.214664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:14.229022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:14.248694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:14.294019 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf1a71-52ea-4382-b456-179bc9977550 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:14.297791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:14.298384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:14.298671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:14.299497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:14.300749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:14.360835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:14.362285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:14.371997 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 979/1951] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:14.519470 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95673ebd60e94d5faa6e46866d94fbba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.520133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95673ebd60e94d5faa6e46866d94fbba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.527791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95673ebd60e94d5faa6e46866d94fbba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.528507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95673ebd60e94d5faa6e46866d94fbba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.575255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:14.576818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:14.592070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d154940-61ee-49d7-aac4-c4eaa1c06ca8 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:14.596520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:14.597269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:14.597724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:14.597999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] PATH_INFO: `/v3/projects/92a5f50e8db24c9b8192e4e6d96990c4` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:14.599317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a5f50e8db24c9b8192e4e6d96990c4)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:14.603136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:14.645076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95673ebd60e94d5faa6e46866d94fbba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.663741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 973/1952] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:14 2022] DELETE /identity/v3/users/95673ebd60e94d5faa6e46866d94fbba => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:14.730765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a5f50e8db24c9b8192e4e6d96990c4', 'request_id': 'req-ff459ab7-667c-4c53-8ca8-e436c7d699d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.833497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a5f50e8db24c9b8192e4e6d96990c4', 'request_id': 'req-ff459ab7-667c-4c53-8ca8-e436c7d699d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.843361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a5f50e8db24c9b8192e4e6d96990c4', 'request_id': 'req-ff459ab7-667c-4c53-8ca8-e436c7d699d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.862139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] The token cache is being invalidate because project 92a5f50e8db24c9b8192e4e6d96990c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:14.862526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.863358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:14.865177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 980/1953] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:14 2022] DELETE /identity/v3/projects/92a5f50e8db24c9b8192e4e6d96990c4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:14.870063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:14.871580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:14.953258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552744ab-ec64-4cf1-b67e-9a64c4982396 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:14.992322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:15.019091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:15.020055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:15.020055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:15.020247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] PATH_INFO: `/v3/users/a023ca9fcca14b03a150a786ca898d24` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:15.021636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a023ca9fcca14b03a150a786ca898d24)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:15.023645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:15.209580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:15.211782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:15.227657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff459ab7-667c-4c53-8ca8-e436c7d699d7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:15.239111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:15.242400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:15.242400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:15.242400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] PATH_INFO: `/v3/projects/e0ccce3cce0141478ac5b1e45acd0cfa` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:15.242400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ccce3cce0141478ac5b1e45acd0cfa)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:15.244371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:15.480478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a023ca9fcca14b03a150a786ca898d24'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.481153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a023ca9fcca14b03a150a786ca898d24'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.491291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a023ca9fcca14b03a150a786ca898d24'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.492051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a023ca9fcca14b03a150a786ca898d24'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.566913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ccce3cce0141478ac5b1e45acd0cfa', 'request_id': 'req-aec0b800-8679-4f32-aadb-273c84fe7970'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.602637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fe2c8650-587f-46fa-b562-db379c0662a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a023ca9fcca14b03a150a786ca898d24'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.623456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 974/1954] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:14 2022] DELETE /identity/v3/users/a023ca9fcca14b03a150a786ca898d24 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:15.698702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ccce3cce0141478ac5b1e45acd0cfa', 'request_id': 'req-aec0b800-8679-4f32-aadb-273c84fe7970'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.706495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ccce3cce0141478ac5b1e45acd0cfa', 'request_id': 'req-aec0b800-8679-4f32-aadb-273c84fe7970'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.751089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] The token cache is being invalidate because project e0ccce3cce0141478ac5b1e45acd0cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:15.751388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.754209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aec0b800-8679-4f32-aadb-273c84fe7970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:15.767188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 981/1955] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:15 2022] DELETE /identity/v3/projects/e0ccce3cce0141478ac5b1e45acd0cfa => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:15.772324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ca0a5-a0d7-403c-8055-da0745ffdf84 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:15.772768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ca0a5-a0d7-403c-8055-da0745ffdf84 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:15.773084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ca0a5-a0d7-403c-8055-da0745ffdf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:15.799886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3d8ca0a5-a0d7-403c-8055-da0745ffdf84 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:15.801218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3d8ca0a5-a0d7-403c-8055-da0745ffdf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:15.861132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 975/1956] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:15.869490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e6d4c-4cb3-4938-86cf-5aa669c0f283 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:15.869770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e6d4c-4cb3-4938-86cf-5aa669c0f283 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:15.870048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e6d4c-4cb3-4938-86cf-5aa669c0f283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:15.906236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3b5e6d4c-4cb3-4938-86cf-5aa669c0f283 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:15.907817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3b5e6d4c-4cb3-4938-86cf-5aa669c0f283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:15.935002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 982/1957] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:15.939383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ca0a5-a0d7-403c-8055-da0745ffdf84 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:15.950145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ca0a5-a0d7-403c-8055-da0745ffdf84 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:15.956068 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:15.956741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:15.957206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:15.957522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:15.958178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:15.958478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:15.960202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:15.972311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 976/1958] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:18:15.976494 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e6d4c-4cb3-4938-86cf-5aa669c0f283 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:15.996319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e6d4c-4cb3-4938-86cf-5aa669c0f283 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.004287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.005081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.005471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.005806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.007849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.010201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.045099 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:16.110721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 983/1959] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:15 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:16.111547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.141009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d830a2-0f91-459d-bc4d-3e46f15c07ba admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.182329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.184627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:16.191244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:16.216671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a77391-2c53-409c-8b96-542f973f0d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.217342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a77391-2c53-409c-8b96-542f973f0d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.217622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a77391-2c53-409c-8b96-542f973f0d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.217892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a77391-2c53-409c-8b96-542f973f0d9f admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.218532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a77391-2c53-409c-8b96-542f973f0d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.220844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a77391-2c53-409c-8b96-542f973f0d9f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.274212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff250-e062-4b71-8cf6-2368f67cca72 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.280079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.280883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.281259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.281636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] PATH_INFO: `/v3/projects/b2b1d41923af4869b4ab8875f1ff7d36` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.283676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b1d41923af4869b4ab8875f1ff7d36)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.285907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.295697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 977/1960] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:16 2022] POST /identity/v3/users => generated 282 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:16.299386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a77391-2c53-409c-8b96-542f973f0d9f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.311090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a77391-2c53-409c-8b96-542f973f0d9f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.331249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b99ac1-21aa-48ad-be2a-a0ffb33e5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.331986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b99ac1-21aa-48ad-be2a-a0ffb33e5514 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.332192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b99ac1-21aa-48ad-be2a-a0ffb33e5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.332463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b99ac1-21aa-48ad-be2a-a0ffb33e5514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.333119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b99ac1-21aa-48ad-be2a-a0ffb33e5514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.334773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b99ac1-21aa-48ad-be2a-a0ffb33e5514 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.347249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 978/1961] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:16.351026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b99ac1-21aa-48ad-be2a-a0ffb33e5514 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.361675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b99ac1-21aa-48ad-be2a-a0ffb33e5514 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.367591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e18aa2-5182-4d35-8264-3dedb898a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.369670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e18aa2-5182-4d35-8264-3dedb898a00d admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.369670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e18aa2-5182-4d35-8264-3dedb898a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.369670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e18aa2-5182-4d35-8264-3dedb898a00d admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.369670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e18aa2-5182-4d35-8264-3dedb898a00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.371825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e18aa2-5182-4d35-8264-3dedb898a00d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.383443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 979/1962] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:16.387287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e18aa2-5182-4d35-8264-3dedb898a00d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.397355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e18aa2-5182-4d35-8264-3dedb898a00d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.402814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5b299-87f9-4f7a-96aa-85c7e387a550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.403499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5b299-87f9-4f7a-96aa-85c7e387a550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.403778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5b299-87f9-4f7a-96aa-85c7e387a550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.404052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5b299-87f9-4f7a-96aa-85c7e387a550 admin admin] PATH_INFO: `/v3/projects/62fb8df98ce94da48cf99e76572bb888/users/63b1d2df125b45279cc6cb5bbe1943a6/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.422283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5b299-87f9-4f7a-96aa-85c7e387a550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62fb8df98ce94da48cf99e76572bb888, user_id=63b1d2df125b45279cc6cb5bbe1943a6, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.427208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5b299-87f9-4f7a-96aa-85c7e387a550 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.435250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b1d41923af4869b4ab8875f1ff7d36', 'request_id': 'req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:16.531247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b1d41923af4869b4ab8875f1ff7d36', 'request_id': 'req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:16.539366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 980/1963] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:16 2022] PUT /identity/v3/projects/62fb8df98ce94da48cf99e76572bb888/users/63b1d2df125b45279cc6cb5bbe1943a6/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:16.540292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b1d41923af4869b4ab8875f1ff7d36', 'request_id': 'req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:16.550625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e99e2-31bd-4bea-9331-34d56505c5d1 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.550972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e99e2-31bd-4bea-9331-34d56505c5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.551290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e99e2-31bd-4bea-9331-34d56505c5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.552459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] The token cache is being invalidate because project b2b1d41923af4869b4ab8875f1ff7d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:16.552916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:16.553763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f5bf3d3b-8c6a-403b-8e8d-0aecd3e74af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:16.559133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 984/1964] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:16 2022] DELETE /identity/v3/projects/b2b1d41923af4869b4ab8875f1ff7d36 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:16.560867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854961de-3b43-458b-97c3-8be9acbbeace None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.562417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854961de-3b43-458b-97c3-8be9acbbeace None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.562994 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854961de-3b43-458b-97c3-8be9acbbeace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.580557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-290e99e2-31bd-4bea-9331-34d56505c5d1 None None] MFA Rules not processed for user `63b1d2df125b45279cc6cb5bbe1943a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:16.582029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-290e99e2-31bd-4bea-9331-34d56505c5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:16.584152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-854961de-3b43-458b-97c3-8be9acbbeace None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:16.586016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-854961de-3b43-458b-97c3-8be9acbbeace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:16.645921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 985/1965] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:16.646695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 981/1966] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:16.651703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854961de-3b43-458b-97c3-8be9acbbeace None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.653210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-854961de-3b43-458b-97c3-8be9acbbeace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:16.654908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33ac84-a93e-499f-969b-a58abf841ace None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.655222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33ac84-a93e-499f-969b-a58abf841ace None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.655492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33ac84-a93e-499f-969b-a58abf841ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.666428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854961de-3b43-458b-97c3-8be9acbbeace None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.670989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f7436-48d1-449c-baf1-7f0a64b59e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.671585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f7436-48d1-449c-baf1-7f0a64b59e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.671858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f7436-48d1-449c-baf1-7f0a64b59e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.672148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f7436-48d1-449c-baf1-7f0a64b59e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.672773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f7436-48d1-449c-baf1-7f0a64b59e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.674660 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f7436-48d1-449c-baf1-7f0a64b59e6f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.675123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-fd33ac84-a93e-499f-969b-a58abf841ace None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:16.676354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fd33ac84-a93e-499f-969b-a58abf841ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:16.684942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 982/1967] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:16.689607 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33ac84-a93e-499f-969b-a58abf841ace None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.699515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33ac84-a93e-499f-969b-a58abf841ace None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.703984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.704592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.704869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.705145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.706053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:16.706360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.707996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.721061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 983/1968] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:18:16.726028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.735643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5deb8-15d7-4663-8f51-293ec4dd34e6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.740101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce1d07-0e20-45e4-a12c-203c5ce4b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.740673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1d07-0e20-45e4-a12c-203c5ce4b174 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.740948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1d07-0e20-45e4-a12c-203c5ce4b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.741212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce1d07-0e20-45e4-a12c-203c5ce4b174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.742333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce1d07-0e20-45e4-a12c-203c5ce4b174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.744238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce1d07-0e20-45e4-a12c-203c5ce4b174 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.797573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 986/1969] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:16 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:16.801625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f7436-48d1-449c-baf1-7f0a64b59e6f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.862779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f7436-48d1-449c-baf1-7f0a64b59e6f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.868014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143162a-baf4-4168-86ed-72027467e417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.868611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143162a-baf4-4168-86ed-72027467e417 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.868932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143162a-baf4-4168-86ed-72027467e417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.869205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143162a-baf4-4168-86ed-72027467e417 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.869819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143162a-baf4-4168-86ed-72027467e417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.871816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143162a-baf4-4168-86ed-72027467e417 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.888608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 984/1970] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:16 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:16.892737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce1d07-0e20-45e4-a12c-203c5ce4b174 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:16.952063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce1d07-0e20-45e4-a12c-203c5ce4b174 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:16.956983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6642-f833-4438-9237-179f3eb75aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:16.957567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6642-f833-4438-9237-179f3eb75aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:16.957846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6642-f833-4438-9237-179f3eb75aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:16.958116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6642-f833-4438-9237-179f3eb75aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:16.958727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d6642-f833-4438-9237-179f3eb75aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:16.960673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d6642-f833-4438-9237-179f3eb75aa0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:16.989992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 987/1971] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:16 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:16.994491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143162a-baf4-4168-86ed-72027467e417 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.004126 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143162a-baf4-4168-86ed-72027467e417 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.009190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83365c17-c836-42cd-8b45-1c294c1a7f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.009816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83365c17-c836-42cd-8b45-1c294c1a7f31 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.010098 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83365c17-c836-42cd-8b45-1c294c1a7f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.010372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83365c17-c836-42cd-8b45-1c294c1a7f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.011061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83365c17-c836-42cd-8b45-1c294c1a7f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.012691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83365c17-c836-42cd-8b45-1c294c1a7f31 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.022620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 988/1972] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:17.029858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83365c17-c836-42cd-8b45-1c294c1a7f31 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.039734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83365c17-c836-42cd-8b45-1c294c1a7f31 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.044400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc08832-0bf9-4d65-88b2-624d03064a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.045007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc08832-0bf9-4d65-88b2-624d03064a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.045288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc08832-0bf9-4d65-88b2-624d03064a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.045582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc08832-0bf9-4d65-88b2-624d03064a27 admin admin] PATH_INFO: `/v3/projects/ce4e1dafbc434ebfb581854de4379f3e/users/37190a9446064f5a80085dd9baca4b4c/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.061313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc08832-0bf9-4d65-88b2-624d03064a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4e1dafbc434ebfb581854de4379f3e, user_id=37190a9446064f5a80085dd9baca4b4c, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.064062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc08832-0bf9-4d65-88b2-624d03064a27 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.088098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 985/1973] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:16 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:17.092200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6642-f833-4438-9237-179f3eb75aa0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.101811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:17.102633 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6642-f833-4438-9237-179f3eb75aa0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.107697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2f3c7-c71f-4b98-8ccd-85c55d0674b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.108289 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2f3c7-c71f-4b98-8ccd-85c55d0674b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.108617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2f3c7-c71f-4b98-8ccd-85c55d0674b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.108893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2f3c7-c71f-4b98-8ccd-85c55d0674b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.109530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2f3c7-c71f-4b98-8ccd-85c55d0674b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.111183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2f3c7-c71f-4b98-8ccd-85c55d0674b9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.122032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 986/1974] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:17.125906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2f3c7-c71f-4b98-8ccd-85c55d0674b9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.135581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2f3c7-c71f-4b98-8ccd-85c55d0674b9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.140287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb07fe7-2850-491c-a5ca-449ed42b9633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.140850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb07fe7-2850-491c-a5ca-449ed42b9633 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.141163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb07fe7-2850-491c-a5ca-449ed42b9633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.141438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb07fe7-2850-491c-a5ca-449ed42b9633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.142081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb07fe7-2850-491c-a5ca-449ed42b9633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.143695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb07fe7-2850-491c-a5ca-449ed42b9633 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.158354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 987/1975] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:17.162573 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb07fe7-2850-491c-a5ca-449ed42b9633 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.162898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 989/1976] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:17 2022] PUT /identity/v3/projects/ce4e1dafbc434ebfb581854de4379f3e/users/37190a9446064f5a80085dd9baca4b4c/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:17.166903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc08832-0bf9-4d65-88b2-624d03064a27 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.175796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb07fe7-2850-491c-a5ca-449ed42b9633 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.223923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:17.237205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc08832-0bf9-4d65-88b2-624d03064a27 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.237388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ff6ab-c91b-477f-8a12-8e75a97102c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.238042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff6ab-c91b-477f-8a12-8e75a97102c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.238318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff6ab-c91b-477f-8a12-8e75a97102c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.238586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff6ab-c91b-477f-8a12-8e75a97102c3 admin admin] PATH_INFO: `/v3/projects/086e5c78010f4bbc944a03bf9b86a17e/users/42c6e15c008847ff9c2481df240f050a/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.241965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9de740-02a1-4606-a65e-e9b7b4334132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.242560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9de740-02a1-4606-a65e-e9b7b4334132 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.242839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9de740-02a1-4606-a65e-e9b7b4334132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.243136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9de740-02a1-4606-a65e-e9b7b4334132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.243775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9de740-02a1-4606-a65e-e9b7b4334132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.245377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9de740-02a1-4606-a65e-e9b7b4334132 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.253628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ff6ab-c91b-477f-8a12-8e75a97102c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=086e5c78010f4bbc944a03bf9b86a17e, user_id=42c6e15c008847ff9c2481df240f050a, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.258571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 990/1977] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:17.258690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ff6ab-c91b-477f-8a12-8e75a97102c3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.259161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9de740-02a1-4606-a65e-e9b7b4334132 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.268554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9de740-02a1-4606-a65e-e9b7b4334132 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.273751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675b36-26e2-47d8-8af3-0779cfbcfe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.274397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675b36-26e2-47d8-8af3-0779cfbcfe4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.274681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675b36-26e2-47d8-8af3-0779cfbcfe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.274973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675b36-26e2-47d8-8af3-0779cfbcfe4c admin admin] PATH_INFO: `/v3/projects/ce4e1dafbc434ebfb581854de4379f3e/users/37190a9446064f5a80085dd9baca4b4c/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.276902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c675b36-26e2-47d8-8af3-0779cfbcfe4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4e1dafbc434ebfb581854de4379f3e, user_id=37190a9446064f5a80085dd9baca4b4c, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.279635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c675b36-26e2-47d8-8af3-0779cfbcfe4c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.290739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 991/1978] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:17 2022] PUT /identity/v3/projects/ce4e1dafbc434ebfb581854de4379f3e/users/37190a9446064f5a80085dd9baca4b4c/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:17.300669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e42930-11ca-48f6-8a51-6793933f090d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.300972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e42930-11ca-48f6-8a51-6793933f090d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.301248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e42930-11ca-48f6-8a51-6793933f090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.328221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e1e42930-11ca-48f6-8a51-6793933f090d None None] MFA Rules not processed for user `37190a9446064f5a80085dd9baca4b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:17.329442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e1e42930-11ca-48f6-8a51-6793933f090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:17.371140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 988/1979] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:17 2022] PUT /identity/v3/projects/086e5c78010f4bbc944a03bf9b86a17e/users/42c6e15c008847ff9c2481df240f050a/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:17.381156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82aeeaa-2cd0-4cc3-94b9-f82b3097902a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.381434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82aeeaa-2cd0-4cc3-94b9-f82b3097902a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.381709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82aeeaa-2cd0-4cc3-94b9-f82b3097902a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.402161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 992/1980] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:17.414988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f82aeeaa-2cd0-4cc3-94b9-f82b3097902a None None] MFA Rules not processed for user `42c6e15c008847ff9c2481df240f050a`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:17.414988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f82aeeaa-2cd0-4cc3-94b9-f82b3097902a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:17.416465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e42930-11ca-48f6-8a51-6793933f090d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.417962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e1e42930-11ca-48f6-8a51-6793933f090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:17.486040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e42930-11ca-48f6-8a51-6793933f090d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.492602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 989/1981] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:17.496774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82aeeaa-2cd0-4cc3-94b9-f82b3097902a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.542692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae036069-7ebe-4b64-b079-c83950cc1c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.543339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036069-7ebe-4b64-b079-c83950cc1c15 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.543615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036069-7ebe-4b64-b079-c83950cc1c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.543885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036069-7ebe-4b64-b079-c83950cc1c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.561373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82aeeaa-2cd0-4cc3-94b9-f82b3097902a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.566054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90705e8-db03-4452-a0fe-5a9958919b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.566648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90705e8-db03-4452-a0fe-5a9958919b6b admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.566923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90705e8-db03-4452-a0fe-5a9958919b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.567217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90705e8-db03-4452-a0fe-5a9958919b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.567826 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90705e8-db03-4452-a0fe-5a9958919b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.569876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90705e8-db03-4452-a0fe-5a9958919b6b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.603837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:17.606079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae036069-7ebe-4b64-b079-c83950cc1c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.607640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae036069-7ebe-4b64-b079-c83950cc1c15 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.617238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 993/1982] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:17.683264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 990/1983] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:17 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:17.687487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae036069-7ebe-4b64-b079-c83950cc1c15 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.698623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae036069-7ebe-4b64-b079-c83950cc1c15 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.764735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cf544-0b6a-449b-9844-c4021b9c4434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.767358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf544-0b6a-449b-9844-c4021b9c4434 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.767852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf544-0b6a-449b-9844-c4021b9c4434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.768395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cf544-0b6a-449b-9844-c4021b9c4434 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.769252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cf544-0b6a-449b-9844-c4021b9c4434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.771416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cf544-0b6a-449b-9844-c4021b9c4434 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.813353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:17.891970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 994/1984] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:17 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:17.905128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90705e8-db03-4452-a0fe-5a9958919b6b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.907436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cf544-0b6a-449b-9844-c4021b9c4434 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.917237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cf544-0b6a-449b-9844-c4021b9c4434 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.921797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12225095-0973-43bb-8e0d-d3e86da395c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.922495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12225095-0973-43bb-8e0d-d3e86da395c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.922776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12225095-0973-43bb-8e0d-d3e86da395c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.923090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12225095-0973-43bb-8e0d-d3e86da395c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.923791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12225095-0973-43bb-8e0d-d3e86da395c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.925386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12225095-0973-43bb-8e0d-d3e86da395c2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:17.935326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 995/1985] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:17.939368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12225095-0973-43bb-8e0d-d3e86da395c2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:17.949702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12225095-0973-43bb-8e0d-d3e86da395c2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.954304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc7f52-944b-4d4a-b245-14e795d23ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.955052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc7f52-944b-4d4a-b245-14e795d23ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.955334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc7f52-944b-4d4a-b245-14e795d23ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.955604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc7f52-944b-4d4a-b245-14e795d23ba3 admin admin] PATH_INFO: `/v3/projects/7517a59b487844c79257dc07fe546367/users/a1adef3b2f1742c0b985acf98a5e2a02/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.962541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90705e8-db03-4452-a0fe-5a9958919b6b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:17.966232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b12c-991e-4c97-9611-d9f175bbdc19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:17.966779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b12c-991e-4c97-9611-d9f175bbdc19 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:17.967096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b12c-991e-4c97-9611-d9f175bbdc19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:17.967369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b12c-991e-4c97-9611-d9f175bbdc19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:17.971670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc7f52-944b-4d4a-b245-14e795d23ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7517a59b487844c79257dc07fe546367, user_id=a1adef3b2f1742c0b985acf98a5e2a02, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:17.974436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc7f52-944b-4d4a-b245-14e795d23ba3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:18.010796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:18.023550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2b12c-991e-4c97-9611-d9f175bbdc19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:18.025674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2b12c-991e-4c97-9611-d9f175bbdc19 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:18.035130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 991/1986] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:18.042767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 996/1987] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:17 2022] PUT /identity/v3/projects/7517a59b487844c79257dc07fe546367/users/a1adef3b2f1742c0b985acf98a5e2a02/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:18.048192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b12c-991e-4c97-9611-d9f175bbdc19 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:18.057545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b12c-991e-4c97-9611-d9f175bbdc19 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:18.116004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403398a-dc45-483c-9edc-3b0979ff6b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:18.116621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403398a-dc45-483c-9edc-3b0979ff6b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:18.116898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403398a-dc45-483c-9edc-3b0979ff6b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:18.117168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403398a-dc45-483c-9edc-3b0979ff6b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:18.117863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403398a-dc45-483c-9edc-3b0979ff6b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:18.119512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403398a-dc45-483c-9edc-3b0979ff6b11 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:18.129888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 992/1988] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:18.134121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc7f52-944b-4d4a-b245-14e795d23ba3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:18.144756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc7f52-944b-4d4a-b245-14e795d23ba3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:18.150170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31ee01-3daa-4b7d-8ef0-2a8ec31581b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:18.150822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ee01-3daa-4b7d-8ef0-2a8ec31581b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:18.151245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ee01-3daa-4b7d-8ef0-2a8ec31581b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:18.151592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ee01-3daa-4b7d-8ef0-2a8ec31581b4 admin admin] PATH_INFO: `/v3/projects/7517a59b487844c79257dc07fe546367/users/a1adef3b2f1742c0b985acf98a5e2a02/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:18.157700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31ee01-3daa-4b7d-8ef0-2a8ec31581b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7517a59b487844c79257dc07fe546367, user_id=a1adef3b2f1742c0b985acf98a5e2a02, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:18.157700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31ee01-3daa-4b7d-8ef0-2a8ec31581b4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:18.166963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 997/1989] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:18 2022] PUT /identity/v3/projects/7517a59b487844c79257dc07fe546367/users/a1adef3b2f1742c0b985acf98a5e2a02/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:18.178287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b25c9-8373-45aa-8965-7258a6d68587 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:18.179741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b25c9-8373-45aa-8965-7258a6d68587 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:18.180028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b25c9-8373-45aa-8965-7258a6d68587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:18.210831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-aa4b25c9-8373-45aa-8965-7258a6d68587 None None] MFA Rules not processed for user `a1adef3b2f1742c0b985acf98a5e2a02`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:18.212118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-aa4b25c9-8373-45aa-8965-7258a6d68587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:18.214099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:18.284039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 993/1990] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:18.594342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:18.598725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31ee01-3daa-4b7d-8ef0-2a8ec31581b4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:18.659002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31ee01-3daa-4b7d-8ef0-2a8ec31581b4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:18.662622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d14db2-ab3a-44ea-87cc-01f2a6c7ecd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:18.663261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14db2-ab3a-44ea-87cc-01f2a6c7ecd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:18.664175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14db2-ab3a-44ea-87cc-01f2a6c7ecd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:18.664459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d14db2-ab3a-44ea-87cc-01f2a6c7ecd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:18.689634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:18.727539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d14db2-ab3a-44ea-87cc-01f2a6c7ecd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:18.728984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d14db2-ab3a-44ea-87cc-01f2a6c7ecd9 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:18.738405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 998/1991] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:20.731149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b25c9-8373-45aa-8965-7258a6d68587 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:20.740927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b25c9-8373-45aa-8965-7258a6d68587 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:20.744532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:20.745129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:20.745436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:20.745702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:20.747488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:20.806991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:20.808467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:20.817436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 994/1992] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:24.761562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d14db2-ab3a-44ea-87cc-01f2a6c7ecd9 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:24.771606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d14db2-ab3a-44ea-87cc-01f2a6c7ecd9 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:24.773865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:24.774470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:24.774747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:24.775052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:24.776803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:24.840753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:24.842192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:24.851633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 999/1993] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:25.034647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:25.094151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:25.106404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca31f16-c101-4ebd-8bfc-bb0d7a493254 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:25.111193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:25.111812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:25.112087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:25.112359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] PATH_INFO: `/v3/users/63b1d2df125b45279cc6cb5bbe1943a6` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:25.113466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b1d2df125b45279cc6cb5bbe1943a6)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:25.115413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:25.374440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b1d2df125b45279cc6cb5bbe1943a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.378069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b1d2df125b45279cc6cb5bbe1943a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.379207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:25.380627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:25.384091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b1d2df125b45279cc6cb5bbe1943a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.384831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b1d2df125b45279cc6cb5bbe1943a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.394341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0766-b233-4caf-90d3-874de1c38daa service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:25.399733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:25.400355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:25.400636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:25.400911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] PATH_INFO: `/v3/users/eb39df0476cd484d8879351b7dfe4822` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:25.402193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb39df0476cd484d8879351b7dfe4822)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:25.404190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:25.563151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-040af507-26e4-47d6-a485-24372d48f4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b1d2df125b45279cc6cb5bbe1943a6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.583743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 995/1994] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:25 2022] DELETE /identity/v3/users/63b1d2df125b45279cc6cb5bbe1943a6 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:25.588130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a683599-bfcc-4f56-8a7d-566b166ff927 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:25.588599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a683599-bfcc-4f56-8a7d-566b166ff927 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:25.588912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a683599-bfcc-4f56-8a7d-566b166ff927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:25.609528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1a683599-bfcc-4f56-8a7d-566b166ff927 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:25.610870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1a683599-bfcc-4f56-8a7d-566b166ff927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:25.677503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 996/1995] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:25.689067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a683599-bfcc-4f56-8a7d-566b166ff927 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:25.690433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1a683599-bfcc-4f56-8a7d-566b166ff927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:25.699747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb39df0476cd484d8879351b7dfe4822'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.710881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb39df0476cd484d8879351b7dfe4822'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.712410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb39df0476cd484d8879351b7dfe4822'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.713568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb39df0476cd484d8879351b7dfe4822'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.730973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb39df0476cd484d8879351b7dfe4822'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:25.764014 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a683599-bfcc-4f56-8a7d-566b166ff927 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:25.769719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97383855-1c04-4d45-8ca3-9882cfdb4479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:25.770366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383855-1c04-4d45-8ca3-9882cfdb4479 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:25.770641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383855-1c04-4d45-8ca3-9882cfdb4479 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:25.770912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383855-1c04-4d45-8ca3-9882cfdb4479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:25.773240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-97383855-1c04-4d45-8ca3-9882cfdb4479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:25.794497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97383855-1c04-4d45-8ca3-9882cfdb4479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:25.794497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97383855-1c04-4d45-8ca3-9882cfdb4479 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:25.805537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 997/1996] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:25.832504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1000/1997] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:25 2022] DELETE /identity/v3/users/eb39df0476cd484d8879351b7dfe4822 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:25.835463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc0f63-a33a-4d00-bf91-4a65774cc917 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:25.835746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc0f63-a33a-4d00-bf91-4a65774cc917 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:25.836048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc0f63-a33a-4d00-bf91-4a65774cc917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:25.864930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-42dc0f63-a33a-4d00-bf91-4a65774cc917 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:25.866569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-42dc0f63-a33a-4d00-bf91-4a65774cc917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:25.880292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:25.939602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 998/1998] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:25.950103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:25.951599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:25.986267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:26.017925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a60b45-6a1f-43b5-addf-753a3836338d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:26.021804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:26.022408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:26.024087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:26.024087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:26.036036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:26.037595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:26.050963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1001/1999] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:26.573588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc0f63-a33a-4d00-bf91-4a65774cc917 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:26.575069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-42dc0f63-a33a-4d00-bf91-4a65774cc917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:26.591325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc0f63-a33a-4d00-bf91-4a65774cc917 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:26.596072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:26.596717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:26.597000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:26.597274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] PATH_INFO: `/v3/projects/62fb8df98ce94da48cf99e76572bb888` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:26.598687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62fb8df98ce94da48cf99e76572bb888)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:26.600539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:26.691458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:26.693766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:26.714676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d6a6a-60d9-4ab5-a154-274e30b0bb96 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:26.723062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:26.724192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:26.724192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:26.726697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] PATH_INFO: `/v3/projects/0c32d78bc2324eb4be4f50683782ce6a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:26.726697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c32d78bc2324eb4be4f50683782ce6a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:26.728380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:26.736933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62fb8df98ce94da48cf99e76572bb888', 'request_id': 'req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:26.742504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62fb8df98ce94da48cf99e76572bb888', 'request_id': 'req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:26.749365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62fb8df98ce94da48cf99e76572bb888', 'request_id': 'req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:26.915798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] The token cache is being invalidate because project 62fb8df98ce94da48cf99e76572bb888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:26.916413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:26.916962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:26.920062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 999/2000] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:26 2022] DELETE /identity/v3/projects/62fb8df98ce94da48cf99e76572bb888 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:26.922584 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:26.924917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:26.935562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c32d78bc2324eb4be4f50683782ce6a', 'request_id': 'req-583d18f4-ede8-4906-a79b-647ac6b7a853'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:26.991382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585e3f6-239c-4c99-97a3-f685e9b9c2d4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:27.073371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:27.074150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:27.074596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:27.074978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] PATH_INFO: `/v3/users/37190a9446064f5a80085dd9baca4b4c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:27.076479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=37190a9446064f5a80085dd9baca4b4c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:27.078597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:27.083486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c32d78bc2324eb4be4f50683782ce6a', 'request_id': 'req-583d18f4-ede8-4906-a79b-647ac6b7a853'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.091417 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c32d78bc2324eb4be4f50683782ce6a', 'request_id': 'req-583d18f4-ede8-4906-a79b-647ac6b7a853'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.105756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] The token cache is being invalidate because project 0c32d78bc2324eb4be4f50683782ce6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:27.106166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.107703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.109386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1002/2001] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:26 2022] DELETE /identity/v3/projects/0c32d78bc2324eb4be4f50683782ce6a => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:27.113409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:27.115155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:27.123978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:27.130329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d18f4-ede8-4906-a79b-647ac6b7a853 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:27.135912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:27.136750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:27.137052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:27.137323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] PATH_INFO: `/v3/users/9a8c403b206844ffa42fdcf97050795a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:27.138498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8c403b206844ffa42fdcf97050795a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:27.140462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:27.428007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37190a9446064f5a80085dd9baca4b4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.428656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37190a9446064f5a80085dd9baca4b4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.436000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37190a9446064f5a80085dd9baca4b4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.436622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37190a9446064f5a80085dd9baca4b4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.496919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8c403b206844ffa42fdcf97050795a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.504718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8c403b206844ffa42fdcf97050795a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.505413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8c403b206844ffa42fdcf97050795a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.506059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8c403b206844ffa42fdcf97050795a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.518617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8c403b206844ffa42fdcf97050795a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.545879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37190a9446064f5a80085dd9baca4b4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:27.564110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1000/2002] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:26 2022] DELETE /identity/v3/users/37190a9446064f5a80085dd9baca4b4c => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:27.712440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1003/2003] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:27 2022] DELETE /identity/v3/users/9a8c403b206844ffa42fdcf97050795a => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:28.157541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:28.162521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:28.262205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a27b6-d3b3-4fcd-a989-23166a8dd10b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:28.322778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:28.323474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.323818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.324182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] PATH_INFO: `/v3/projects/ce4e1dafbc434ebfb581854de4379f3e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.325414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4e1dafbc434ebfb581854de4379f3e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:28.327517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:28.362517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:28.364122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:28.379434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1966cf-394d-43c2-b9e7-0b7fad371b0d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:28.385237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:28.385993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.386378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.386790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] PATH_INFO: `/v3/projects/3eaa6e02f70d40eaa84f1bdeb2cd95d9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.388178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eaa6e02f70d40eaa84f1bdeb2cd95d9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:28.390472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:28.417312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:28.472331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4e1dafbc434ebfb581854de4379f3e', 'request_id': 'req-6a78e267-311c-4579-a83c-36c9cadd7566'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.477800 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4e1dafbc434ebfb581854de4379f3e', 'request_id': 'req-6a78e267-311c-4579-a83c-36c9cadd7566'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.484211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4e1dafbc434ebfb581854de4379f3e', 'request_id': 'req-6a78e267-311c-4579-a83c-36c9cadd7566'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.592681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eaa6e02f70d40eaa84f1bdeb2cd95d9', 'request_id': 'req-3b6c135f-d244-47a5-9ccd-268d2d62524c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.619766 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] The token cache is being invalidate because project ce4e1dafbc434ebfb581854de4379f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:28.625940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.625940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6a78e267-311c-4579-a83c-36c9cadd7566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.625940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1001/2004] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:28 2022] DELETE /identity/v3/projects/ce4e1dafbc434ebfb581854de4379f3e => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:28.628673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7775e2b-0018-4e02-af22-4a791517259a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.628970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7775e2b-0018-4e02-af22-4a791517259a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.629243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7775e2b-0018-4e02-af22-4a791517259a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.650722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c7775e2b-0018-4e02-af22-4a791517259a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:28.654623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c7775e2b-0018-4e02-af22-4a791517259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:28.709956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1002/2005] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:28.717690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfedbf3-c83b-4905-828c-4eb7bbe80876 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.718016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfedbf3-c83b-4905-828c-4eb7bbe80876 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.718290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfedbf3-c83b-4905-828c-4eb7bbe80876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.739860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3dfedbf3-c83b-4905-828c-4eb7bbe80876 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:28.740270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eaa6e02f70d40eaa84f1bdeb2cd95d9', 'request_id': 'req-3b6c135f-d244-47a5-9ccd-268d2d62524c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.741957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3dfedbf3-c83b-4905-828c-4eb7bbe80876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:28.747970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eaa6e02f70d40eaa84f1bdeb2cd95d9', 'request_id': 'req-3b6c135f-d244-47a5-9ccd-268d2d62524c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.754348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1003/2006] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:28.763218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] The token cache is being invalidate because project 3eaa6e02f70d40eaa84f1bdeb2cd95d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:28.763218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.764624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b6c135f-d244-47a5-9ccd-268d2d62524c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:28.766036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfedbf3-c83b-4905-828c-4eb7bbe80876 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:28.766974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1004/2007] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:28 2022] DELETE /identity/v3/projects/3eaa6e02f70d40eaa84f1bdeb2cd95d9 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:28.770965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b08dad-16e6-4378-9818-e95fe8663be1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.771393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b08dad-16e6-4378-9818-e95fe8663be1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.771745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b08dad-16e6-4378-9818-e95fe8663be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.775596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfedbf3-c83b-4905-828c-4eb7bbe80876 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:28.786458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:28.787188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.787510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.787787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.788444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:28.788751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:28.790396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:28.793066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-96b08dad-16e6-4378-9818-e95fe8663be1 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:28.794377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-96b08dad-16e6-4378-9818-e95fe8663be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:28.803179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1004/2008] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:18:28.809935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1005/2009] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:28.810106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:28.810106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:28.811996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3662a-8c7a-486d-96da-f24b0eb8bbaa None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.812298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3662a-8c7a-486d-96da-f24b0eb8bbaa None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.812535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3662a-8c7a-486d-96da-f24b0eb8bbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.823906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39007-a2b2-482f-b9d9-1a815ffc5580 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:28.828794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a00f62-4836-4279-963f-3ab0129b4884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:28.829400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a00f62-4836-4279-963f-3ab0129b4884 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.829684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a00f62-4836-4279-963f-3ab0129b4884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.829958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a00f62-4836-4279-963f-3ab0129b4884 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.834296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a00f62-4836-4279-963f-3ab0129b4884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:28.834296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a00f62-4836-4279-963f-3ab0129b4884 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:28.834473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dce3662a-8c7a-486d-96da-f24b0eb8bbaa None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:28.834473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dce3662a-8c7a-486d-96da-f24b0eb8bbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:28.842533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1006/2010] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:28.846873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3662a-8c7a-486d-96da-f24b0eb8bbaa None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:28.856767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3662a-8c7a-486d-96da-f24b0eb8bbaa None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:28.861614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:28.862319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.862698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.863057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.863709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:28.864014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:28.865659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:28.878586 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1007/2011] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:18:28.883111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:28.892710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e70496-2f0e-4b07-a181-c15e17e2c0cf admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:28.897487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40233b-c900-485f-8cf7-687b7bda05e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:28.898074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40233b-c900-485f-8cf7-687b7bda05e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:28.898360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40233b-c900-485f-8cf7-687b7bda05e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:28.898710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40233b-c900-485f-8cf7-687b7bda05e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:28.899313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c40233b-c900-485f-8cf7-687b7bda05e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:28.901355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c40233b-c900-485f-8cf7-687b7bda05e0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:28.930833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1005/2012] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:28 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:28.934799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a00f62-4836-4279-963f-3ab0129b4884 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:28.995983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a00f62-4836-4279-963f-3ab0129b4884 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.000647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd318046-ba90-4c89-95f7-858fab7067c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.001246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd318046-ba90-4c89-95f7-858fab7067c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.001532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd318046-ba90-4c89-95f7-858fab7067c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.001804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd318046-ba90-4c89-95f7-858fab7067c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.002418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd318046-ba90-4c89-95f7-858fab7067c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.004384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd318046-ba90-4c89-95f7-858fab7067c8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.042806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1008/2013] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:28 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:29.046807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40233b-c900-485f-8cf7-687b7bda05e0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.107291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40233b-c900-485f-8cf7-687b7bda05e0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.111995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315b9b8-beb3-467f-9181-6464badacccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.112626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315b9b8-beb3-467f-9181-6464badacccc admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.112914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315b9b8-beb3-467f-9181-6464badacccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.113191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315b9b8-beb3-467f-9181-6464badacccc admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.113814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315b9b8-beb3-467f-9181-6464badacccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.115843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315b9b8-beb3-467f-9181-6464badacccc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.140582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1006/2014] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:28 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:29.144557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd318046-ba90-4c89-95f7-858fab7067c8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.153906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd318046-ba90-4c89-95f7-858fab7067c8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.158634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a53ba8-d047-448f-a1b1-971a1fbe8e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.159252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a53ba8-d047-448f-a1b1-971a1fbe8e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.159517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a53ba8-d047-448f-a1b1-971a1fbe8e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.159804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a53ba8-d047-448f-a1b1-971a1fbe8e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.160415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a53ba8-d047-448f-a1b1-971a1fbe8e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.162005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a53ba8-d047-448f-a1b1-971a1fbe8e88 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.172180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1007/2015] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:29.175769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a53ba8-d047-448f-a1b1-971a1fbe8e88 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.185549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a53ba8-d047-448f-a1b1-971a1fbe8e88 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.190146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c8c24-3dd2-4bbe-afab-48858d6c2335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.190691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c8c24-3dd2-4bbe-afab-48858d6c2335 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.191006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c8c24-3dd2-4bbe-afab-48858d6c2335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.191283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c8c24-3dd2-4bbe-afab-48858d6c2335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.191883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c8c24-3dd2-4bbe-afab-48858d6c2335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.193408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c8c24-3dd2-4bbe-afab-48858d6c2335 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.204680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1008/2016] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:29.219225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c8c24-3dd2-4bbe-afab-48858d6c2335 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.233403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c8c24-3dd2-4bbe-afab-48858d6c2335 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.241691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de76bb-5870-4eb6-8d25-a771f6544dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.242365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de76bb-5870-4eb6-8d25-a771f6544dde admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.242662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de76bb-5870-4eb6-8d25-a771f6544dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.242935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de76bb-5870-4eb6-8d25-a771f6544dde admin admin] PATH_INFO: `/v3/projects/b28425f874fb4b00b99a670120c59903/users/6dc8bb455f124475b1c053788e05c55c/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.264224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1009/2017] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:29 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:29.265177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315b9b8-beb3-467f-9181-6464badacccc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.268242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de76bb-5870-4eb6-8d25-a771f6544dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28425f874fb4b00b99a670120c59903, user_id=6dc8bb455f124475b1c053788e05c55c, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.270920 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de76bb-5870-4eb6-8d25-a771f6544dde admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.281241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315b9b8-beb3-467f-9181-6464badacccc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.287522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1c89a-75be-4cd0-8871-688413fb59ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.288418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c89a-75be-4cd0-8871-688413fb59ed admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.288939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c89a-75be-4cd0-8871-688413fb59ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.290380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c89a-75be-4cd0-8871-688413fb59ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.293466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1c89a-75be-4cd0-8871-688413fb59ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.293466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1c89a-75be-4cd0-8871-688413fb59ed admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.305082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1010/2018] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:29.310072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1c89a-75be-4cd0-8871-688413fb59ed admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.321335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1c89a-75be-4cd0-8871-688413fb59ed admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.326913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e8f83-b0ea-42f9-b1a8-224e42eb5563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.327822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e8f83-b0ea-42f9-b1a8-224e42eb5563 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.333500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e8f83-b0ea-42f9-b1a8-224e42eb5563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.333500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e8f83-b0ea-42f9-b1a8-224e42eb5563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.333500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e8f83-b0ea-42f9-b1a8-224e42eb5563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.333500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e8f83-b0ea-42f9-b1a8-224e42eb5563 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.349703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1011/2019] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:29.351146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e8f83-b0ea-42f9-b1a8-224e42eb5563 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.375700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e8f83-b0ea-42f9-b1a8-224e42eb5563 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.383642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1009/2020] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:29 2022] PUT /identity/v3/projects/b28425f874fb4b00b99a670120c59903/users/6dc8bb455f124475b1c053788e05c55c/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:29.389972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a298faf-9a90-45ff-bdf4-986fb9978f96 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.390250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a298faf-9a90-45ff-bdf4-986fb9978f96 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.390523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a298faf-9a90-45ff-bdf4-986fb9978f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.423795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4a298faf-9a90-45ff-bdf4-986fb9978f96 None None] MFA Rules not processed for user `6dc8bb455f124475b1c053788e05c55c`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:29.425031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4a298faf-9a90-45ff-bdf4-986fb9978f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:29.449744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2993c82-aff7-43b7-8bc8-862d450085f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.450777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2993c82-aff7-43b7-8bc8-862d450085f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.451153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2993c82-aff7-43b7-8bc8-862d450085f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.451557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2993c82-aff7-43b7-8bc8-862d450085f7 admin admin] PATH_INFO: `/v3/projects/4af4f8dc586e41a08c1781d7832ec85a/users/e587cb3ed6cc45409450dc1b32b55726/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.468557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2993c82-aff7-43b7-8bc8-862d450085f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af4f8dc586e41a08c1781d7832ec85a, user_id=e587cb3ed6cc45409450dc1b32b55726, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.471594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2993c82-aff7-43b7-8bc8-862d450085f7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.497099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1010/2021] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:29.501702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a298faf-9a90-45ff-bdf4-986fb9978f96 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.512452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a298faf-9a90-45ff-bdf4-986fb9978f96 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.517283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275e972-c2bb-44d6-ac5e-ed7871fe93da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.517882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275e972-c2bb-44d6-ac5e-ed7871fe93da admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.518165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275e972-c2bb-44d6-ac5e-ed7871fe93da admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.518439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275e972-c2bb-44d6-ac5e-ed7871fe93da admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.519092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275e972-c2bb-44d6-ac5e-ed7871fe93da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.521055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275e972-c2bb-44d6-ac5e-ed7871fe93da admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.563051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1012/2022] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:29 2022] PUT /identity/v3/projects/4af4f8dc586e41a08c1781d7832ec85a/users/e587cb3ed6cc45409450dc1b32b55726/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:29.574210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3a5b9-e36c-40f4-99c3-4fd3d2a3cb94 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.574500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3a5b9-e36c-40f4-99c3-4fd3d2a3cb94 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.574777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3a5b9-e36c-40f4-99c3-4fd3d2a3cb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.602758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f2a3a5b9-e36c-40f4-99c3-4fd3d2a3cb94 None None] MFA Rules not processed for user `e587cb3ed6cc45409450dc1b32b55726`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:29.604031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f2a3a5b9-e36c-40f4-99c3-4fd3d2a3cb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:29.652330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1011/2023] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:29 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:29.656451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275e972-c2bb-44d6-ac5e-ed7871fe93da admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.671927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1013/2024] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:29.676231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3a5b9-e36c-40f4-99c3-4fd3d2a3cb94 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.717308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275e972-c2bb-44d6-ac5e-ed7871fe93da admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.722432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6edcbc-5047-4061-ba7a-4c825e3dac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.723101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edcbc-5047-4061-ba7a-4c825e3dac3c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.728358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edcbc-5047-4061-ba7a-4c825e3dac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.728358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edcbc-5047-4061-ba7a-4c825e3dac3c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.728358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6edcbc-5047-4061-ba7a-4c825e3dac3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.728358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6edcbc-5047-4061-ba7a-4c825e3dac3c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.737550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3a5b9-e36c-40f4-99c3-4fd3d2a3cb94 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.742305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9707f-8c68-49d3-b986-94368062f4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.742917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9707f-8c68-49d3-b986-94368062f4bb admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.743225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9707f-8c68-49d3-b986-94368062f4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.743493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9707f-8c68-49d3-b986-94368062f4bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.744124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9707f-8c68-49d3-b986-94368062f4bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.746364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9707f-8c68-49d3-b986-94368062f4bb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.863642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1012/2025] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:29 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:29.867441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6edcbc-5047-4061-ba7a-4c825e3dac3c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.875819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1014/2026] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:29 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:29.876777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9707f-8c68-49d3-b986-94368062f4bb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.877626 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6edcbc-5047-4061-ba7a-4c825e3dac3c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.939326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ab700-2688-486c-8fa6-8930f9c4c993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.939326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ab700-2688-486c-8fa6-8930f9c4c993 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.939326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ab700-2688-486c-8fa6-8930f9c4c993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.939326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ab700-2688-486c-8fa6-8930f9c4c993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.939705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9707f-8c68-49d3-b986-94368062f4bb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.939801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ab700-2688-486c-8fa6-8930f9c4c993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.940754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ab700-2688-486c-8fa6-8930f9c4c993 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.944810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a5c3-e700-4328-8c12-886648dafcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.945608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a5c3-e700-4328-8c12-886648dafcf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.945961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a5c3-e700-4328-8c12-886648dafcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.946358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a5c3-e700-4328-8c12-886648dafcf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.947191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920a5c3-e700-4328-8c12-886648dafcf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.949345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920a5c3-e700-4328-8c12-886648dafcf7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:29.951870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1013/2027] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:29.955498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ab700-2688-486c-8fa6-8930f9c4c993 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:29.968441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ab700-2688-486c-8fa6-8930f9c4c993 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:29.973409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e16fc-b2f6-49e7-a51d-677dc57ff825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:29.974203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e16fc-b2f6-49e7-a51d-677dc57ff825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:29.974560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e16fc-b2f6-49e7-a51d-677dc57ff825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:29.974833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e16fc-b2f6-49e7-a51d-677dc57ff825 admin admin] PATH_INFO: `/v3/projects/6389dab63b984b8181cf0b35aa30db62/users/90de5ccbf073420ebc29e25205e98659/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:29.993616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e16fc-b2f6-49e7-a51d-677dc57ff825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6389dab63b984b8181cf0b35aa30db62, user_id=90de5ccbf073420ebc29e25205e98659, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:29.996537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e16fc-b2f6-49e7-a51d-677dc57ff825 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.047441 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1015/2028] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:29 2022] POST /identity/v3/users => generated 296 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:30.048659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a5c3-e700-4328-8c12-886648dafcf7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.058521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a5c3-e700-4328-8c12-886648dafcf7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.064309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4113034-fb51-445c-9063-0a07a230c738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.065040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4113034-fb51-445c-9063-0a07a230c738 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.065320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4113034-fb51-445c-9063-0a07a230c738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.065590 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4113034-fb51-445c-9063-0a07a230c738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.066243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4113034-fb51-445c-9063-0a07a230c738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.067883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4113034-fb51-445c-9063-0a07a230c738 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.078422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1016/2029] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:30.083303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4113034-fb51-445c-9063-0a07a230c738 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.093840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4113034-fb51-445c-9063-0a07a230c738 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.098454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fa7f1-cdbb-4ee2-81f4-3e8b82a8cff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.099247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fa7f1-cdbb-4ee2-81f4-3e8b82a8cff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.099524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fa7f1-cdbb-4ee2-81f4-3e8b82a8cff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.099801 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fa7f1-cdbb-4ee2-81f4-3e8b82a8cff0 admin admin] PATH_INFO: `/v3/projects/e7f818ce13d54730a48d376fee836e87/users/85c6404ff25741d0aa8256d95695fb74/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.113757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1014/2030] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:29 2022] PUT /identity/v3/projects/6389dab63b984b8181cf0b35aa30db62/users/90de5ccbf073420ebc29e25205e98659/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:30.115572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fa7f1-cdbb-4ee2-81f4-3e8b82a8cff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f818ce13d54730a48d376fee836e87, user_id=85c6404ff25741d0aa8256d95695fb74, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.117841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e16fc-b2f6-49e7-a51d-677dc57ff825 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.118257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fa7f1-cdbb-4ee2-81f4-3e8b82a8cff0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.176837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e16fc-b2f6-49e7-a51d-677dc57ff825 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.181613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f6419-c351-4e19-b9a7-250690ff5226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.182205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f6419-c351-4e19-b9a7-250690ff5226 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.182481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f6419-c351-4e19-b9a7-250690ff5226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.182753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f6419-c351-4e19-b9a7-250690ff5226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.183415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f6419-c351-4e19-b9a7-250690ff5226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.185011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f6419-c351-4e19-b9a7-250690ff5226 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.194594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1015/2031] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:30.198714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f6419-c351-4e19-b9a7-250690ff5226 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.208167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f6419-c351-4e19-b9a7-250690ff5226 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.215533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1017/2032] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:30 2022] PUT /identity/v3/projects/e7f818ce13d54730a48d376fee836e87/users/85c6404ff25741d0aa8256d95695fb74/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:30.219637 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fa7f1-cdbb-4ee2-81f4-3e8b82a8cff0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.266408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9381390-2161-4e58-9f27-0f57231250ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.267140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381390-2161-4e58-9f27-0f57231250ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.267452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381390-2161-4e58-9f27-0f57231250ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.267728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381390-2161-4e58-9f27-0f57231250ab admin admin] PATH_INFO: `/v3/projects/6389dab63b984b8181cf0b35aa30db62/users/90de5ccbf073420ebc29e25205e98659/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.269958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9381390-2161-4e58-9f27-0f57231250ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=6389dab63b984b8181cf0b35aa30db62, user_id=90de5ccbf073420ebc29e25205e98659, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.272689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9381390-2161-4e58-9f27-0f57231250ab admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.281957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fa7f1-cdbb-4ee2-81f4-3e8b82a8cff0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.283756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1016/2033] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:30 2022] PUT /identity/v3/projects/6389dab63b984b8181cf0b35aa30db62/users/90de5ccbf073420ebc29e25205e98659/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:30.292429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:30.299476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253837-04af-4437-9d4a-ce095ac72510 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.299856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253837-04af-4437-9d4a-ce095ac72510 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.300234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253837-04af-4437-9d4a-ce095ac72510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.327531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9c253837-04af-4437-9d4a-ce095ac72510 None None] MFA Rules not processed for user `90de5ccbf073420ebc29e25205e98659`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:30.328798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9c253837-04af-4437-9d4a-ce095ac72510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:30.346627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a20353-bb7e-4841-a38f-228cdc49e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.347273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a20353-bb7e-4841-a38f-228cdc49e82a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.347557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a20353-bb7e-4841-a38f-228cdc49e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.347832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a20353-bb7e-4841-a38f-228cdc49e82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.348475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a20353-bb7e-4841-a38f-228cdc49e82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.350167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a20353-bb7e-4841-a38f-228cdc49e82a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.360458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1018/2034] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:30.364854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a20353-bb7e-4841-a38f-228cdc49e82a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.375265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a20353-bb7e-4841-a38f-228cdc49e82a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.380258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daab35a-c065-4075-b358-b05768d2bb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.380914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daab35a-c065-4075-b358-b05768d2bb79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.381195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daab35a-c065-4075-b358-b05768d2bb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.381467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daab35a-c065-4075-b358-b05768d2bb79 admin admin] PATH_INFO: `/v3/projects/e7f818ce13d54730a48d376fee836e87/users/85c6404ff25741d0aa8256d95695fb74/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.383944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daab35a-c065-4075-b358-b05768d2bb79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f818ce13d54730a48d376fee836e87, user_id=85c6404ff25741d0aa8256d95695fb74, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.386581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daab35a-c065-4075-b358-b05768d2bb79 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.400073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1019/2035] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:30 2022] PUT /identity/v3/projects/e7f818ce13d54730a48d376fee836e87/users/85c6404ff25741d0aa8256d95695fb74/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:30.411995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1017/2036] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:30.418561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0046ce-8687-431f-bcdc-a01d87ede519 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.418561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0046ce-8687-431f-bcdc-a01d87ede519 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.418561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0046ce-8687-431f-bcdc-a01d87ede519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.426226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253837-04af-4437-9d4a-ce095ac72510 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.427715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9c253837-04af-4437-9d4a-ce095ac72510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:30.450312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0d0046ce-8687-431f-bcdc-a01d87ede519 None None] MFA Rules not processed for user `85c6404ff25741d0aa8256d95695fb74`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:30.451606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0d0046ce-8687-431f-bcdc-a01d87ede519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:30.493435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253837-04af-4437-9d4a-ce095ac72510 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.529646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1020/2037] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:30.541429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0046ce-8687-431f-bcdc-a01d87ede519 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.542905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0d0046ce-8687-431f-bcdc-a01d87ede519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:30.551513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00438a0d-773c-4378-9d68-395f67c8d88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.552131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00438a0d-773c-4378-9d68-395f67c8d88d service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.552415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00438a0d-773c-4378-9d68-395f67c8d88d service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.552688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00438a0d-773c-4378-9d68-395f67c8d88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.603874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0046ce-8687-431f-bcdc-a01d87ede519 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.607515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e810-5f32-4382-b83f-a491850fd6d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.608117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e810-5f32-4382-b83f-a491850fd6d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.608389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e810-5f32-4382-b83f-a491850fd6d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.608654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e810-5f32-4382-b83f-a491850fd6d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.615869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00438a0d-773c-4378-9d68-395f67c8d88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.617344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00438a0d-773c-4378-9d68-395f67c8d88d service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.617481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e810-5f32-4382-b83f-a491850fd6d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.619662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e810-5f32-4382-b83f-a491850fd6d6 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.625053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:30.627958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1018/2038] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:30.631251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1021/2039] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:30.732506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00438a0d-773c-4378-9d68-395f67c8d88d service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.741326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00438a0d-773c-4378-9d68-395f67c8d88d service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.746169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb727d6-5265-46b6-acf9-bc53274ad7c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.746775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb727d6-5265-46b6-acf9-bc53274ad7c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.747086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb727d6-5265-46b6-acf9-bc53274ad7c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.750292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb727d6-5265-46b6-acf9-bc53274ad7c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.763819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e810-5f32-4382-b83f-a491850fd6d6 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:30.772351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e810-5f32-4382-b83f-a491850fd6d6 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:30.776820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71d240-1f03-424a-8c4f-102780d98de0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:30.777754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71d240-1f03-424a-8c4f-102780d98de0 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:30.777754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71d240-1f03-424a-8c4f-102780d98de0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:30.777971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71d240-1f03-424a-8c4f-102780d98de0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:30.809819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb727d6-5265-46b6-acf9-bc53274ad7c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.811349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb727d6-5265-46b6-acf9-bc53274ad7c3 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.820257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1019/2040] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:30.842324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71d240-1f03-424a-8c4f-102780d98de0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:30.842324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71d240-1f03-424a-8c4f-102780d98de0 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:30.850329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1022/2041] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:31.103342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:31.301045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:31.581699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb727d6-5265-46b6-acf9-bc53274ad7c3 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:31.645631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb727d6-5265-46b6-acf9-bc53274ad7c3 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:31.650155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:31.650784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:31.651097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:31.651370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] PATH_INFO: `/v3/users/e587cb3ed6cc45409450dc1b32b55726` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:31.652498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e587cb3ed6cc45409450dc1b32b55726)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:31.654473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:31.930771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e587cb3ed6cc45409450dc1b32b55726'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:31.931465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e587cb3ed6cc45409450dc1b32b55726'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:31.938317 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e587cb3ed6cc45409450dc1b32b55726'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:31.938970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e587cb3ed6cc45409450dc1b32b55726'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:32.054811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e587cb3ed6cc45409450dc1b32b55726'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:32.073158 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1020/2042] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:31 2022] DELETE /identity/v3/users/e587cb3ed6cc45409450dc1b32b55726 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:32.077632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015045f-451f-4fba-b8f0-eabbcf356818 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:32.078137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015045f-451f-4fba-b8f0-eabbcf356818 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:32.078246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015045f-451f-4fba-b8f0-eabbcf356818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:32.097589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9015045f-451f-4fba-b8f0-eabbcf356818 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:32.098788 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9015045f-451f-4fba-b8f0-eabbcf356818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:32.145073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:32.159715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1023/2043] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:32.169454 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:32.170903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:32.216446 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:32.229887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c318c7-548d-4ba2-b402-77fdbbbd3ed7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:32.234106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:32.234709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:32.235023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:32.235296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:32.244761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:32.246189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:32.255968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1021/2044] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:32.366562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015045f-451f-4fba-b8f0-eabbcf356818 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:32.368087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9015045f-451f-4fba-b8f0-eabbcf356818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:32.427237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015045f-451f-4fba-b8f0-eabbcf356818 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:32.431473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:32.432105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:32.432389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:32.432669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:32.434881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:32.504205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:32.506084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:32.517893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1024/2045] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:32.784236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:32.942380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:32.948188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:32.949592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:32.964214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2a0ab-2409-48bf-945a-e49a43974807 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:32.969117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:32.969702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:32.969982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:32.970257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] PATH_INFO: `/v3/projects/4af4f8dc586e41a08c1781d7832ec85a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:32.971678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af4f8dc586e41a08c1781d7832ec85a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:32.973551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:33.098697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af4f8dc586e41a08c1781d7832ec85a', 'request_id': 'req-818014f0-4550-4538-bb0a-78e94906e5cb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.104372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af4f8dc586e41a08c1781d7832ec85a', 'request_id': 'req-818014f0-4550-4538-bb0a-78e94906e5cb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.110493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af4f8dc586e41a08c1781d7832ec85a', 'request_id': 'req-818014f0-4550-4538-bb0a-78e94906e5cb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.282842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] The token cache is being invalidate because project 4af4f8dc586e41a08c1781d7832ec85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:33.283162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.283788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.285328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1022/2046] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:32 2022] DELETE /identity/v3/projects/4af4f8dc586e41a08c1781d7832ec85a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:33.289368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:33.299707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f7ed5-ed7e-4648-ae62-f6ab3837ed33 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:33.360563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:33.361195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:33.361473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:33.361745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] PATH_INFO: `/v3/users/85c6404ff25741d0aa8256d95695fb74` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:33.362925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c6404ff25741d0aa8256d95695fb74)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:33.364911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:33.608699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:33.622436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c6404ff25741d0aa8256d95695fb74'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.629390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c6404ff25741d0aa8256d95695fb74'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.630066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c6404ff25741d0aa8256d95695fb74'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.630697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c6404ff25741d0aa8256d95695fb74'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.642858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5c7d9e24-d922-4172-ac16-65ff247eae8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c6404ff25741d0aa8256d95695fb74'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:33.776212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1025/2047] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:33 2022] DELETE /identity/v3/users/85c6404ff25741d0aa8256d95695fb74 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:33.982687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:34.357459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:34.358889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:34.374762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818014f0-4550-4538-bb0a-78e94906e5cb admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:34.439107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:34.439746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:34.441262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:34.441262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] PATH_INFO: `/v3/projects/e7f818ce13d54730a48d376fee836e87` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:34.441489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f818ce13d54730a48d376fee836e87)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:34.443479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:34.598248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f818ce13d54730a48d376fee836e87', 'request_id': 'req-2842786a-382c-496b-90e4-9d1122d6fddd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:34.604205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f818ce13d54730a48d376fee836e87', 'request_id': 'req-2842786a-382c-496b-90e4-9d1122d6fddd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:34.610277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f818ce13d54730a48d376fee836e87', 'request_id': 'req-2842786a-382c-496b-90e4-9d1122d6fddd'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:34.615442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:34.737694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] The token cache is being invalidate because project e7f818ce13d54730a48d376fee836e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:34.737983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:34.738616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2842786a-382c-496b-90e4-9d1122d6fddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:34.740129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1023/2048] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:34 2022] DELETE /identity/v3/projects/e7f818ce13d54730a48d376fee836e87 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:34.744908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccd6b6-0195-4f7b-8690-37e07463721c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:34.745390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccd6b6-0195-4f7b-8690-37e07463721c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:34.745479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccd6b6-0195-4f7b-8690-37e07463721c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:34.764623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7eccd6b6-0195-4f7b-8690-37e07463721c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:34.765807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7eccd6b6-0195-4f7b-8690-37e07463721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:34.824549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1026/2049] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:34.832320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225102-c6f2-403a-ae79-e816889d3da6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:34.832600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225102-c6f2-403a-ae79-e816889d3da6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:34.832870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225102-c6f2-403a-ae79-e816889d3da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:34.853195 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9c225102-c6f2-403a-ae79-e816889d3da6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:34.854372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9c225102-c6f2-403a-ae79-e816889d3da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:34.862086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1024/2050] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:34.866221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccd6b6-0195-4f7b-8690-37e07463721c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:34.877042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccd6b6-0195-4f7b-8690-37e07463721c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:34.882159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:34.882799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:34.883111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:34.883389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:34.884656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:34.884962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:34.886570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:34.900129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1027/2051] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:18:34.904337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225102-c6f2-403a-ae79-e816889d3da6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:34.913600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225102-c6f2-403a-ae79-e816889d3da6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:34.918541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3c31f-0988-4288-b0bc-3aefe5d4adf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:34.919196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c31f-0988-4288-b0bc-3aefe5d4adf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:34.919472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c31f-0988-4288-b0bc-3aefe5d4adf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:34.919742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c31f-0988-4288-b0bc-3aefe5d4adf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:34.920456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3c31f-0988-4288-b0bc-3aefe5d4adf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:34.922458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3c31f-0988-4288-b0bc-3aefe5d4adf9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:35.045523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1025/2052] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:34 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:35.049652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:35.060293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386e85c-ef9c-4bdb-a42f-bea4e9ef2025 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:35.115240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc585a6-5473-4b39-a788-d7bcc1ac8cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:35.115832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc585a6-5473-4b39-a788-d7bcc1ac8cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.116108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc585a6-5473-4b39-a788-d7bcc1ac8cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.116384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc585a6-5473-4b39-a788-d7bcc1ac8cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.117012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc585a6-5473-4b39-a788-d7bcc1ac8cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:35.119600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc585a6-5473-4b39-a788-d7bcc1ac8cd0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:35.219128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1028/2053] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:35 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:35.223481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3c31f-0988-4288-b0bc-3aefe5d4adf9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:35.236294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3c31f-0988-4288-b0bc-3aefe5d4adf9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:35.241379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05dae2-9095-4b9d-bce9-114e5d99d8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:35.241985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05dae2-9095-4b9d-bce9-114e5d99d8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.242261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05dae2-9095-4b9d-bce9-114e5d99d8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.242527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05dae2-9095-4b9d-bce9-114e5d99d8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.243693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05dae2-9095-4b9d-bce9-114e5d99d8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:35.245345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05dae2-9095-4b9d-bce9-114e5d99d8b0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:35.256012 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1026/2054] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:35.260661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc585a6-5473-4b39-a788-d7bcc1ac8cd0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:35.272186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc585a6-5473-4b39-a788-d7bcc1ac8cd0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:35.277301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95e0c0-5b6b-4731-ace2-dcca44b6c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:35.277911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e0c0-5b6b-4731-ace2-dcca44b6c5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.278178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e0c0-5b6b-4731-ace2-dcca44b6c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.278445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95e0c0-5b6b-4731-ace2-dcca44b6c5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.279410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95e0c0-5b6b-4731-ace2-dcca44b6c5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:35.280775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95e0c0-5b6b-4731-ace2-dcca44b6c5eb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:35.293091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1029/2055] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:35.295588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:35.308436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05dae2-9095-4b9d-bce9-114e5d99d8b0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:35.318641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05dae2-9095-4b9d-bce9-114e5d99d8b0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:35.324333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b665c1-2d6b-486a-97a0-6398063d934b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:35.325053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b665c1-2d6b-486a-97a0-6398063d934b admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.325525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b665c1-2d6b-486a-97a0-6398063d934b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.325815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b665c1-2d6b-486a-97a0-6398063d934b admin admin] PATH_INFO: `/v3/projects/5183d9ba6cb340b4a5c7ce03162c6e64/users/650ab868bb9c4d63b4fff14936ceee0f/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.341671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b665c1-2d6b-486a-97a0-6398063d934b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5183d9ba6cb340b4a5c7ce03162c6e64, user_id=650ab868bb9c4d63b4fff14936ceee0f, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:35.344396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b665c1-2d6b-486a-97a0-6398063d934b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:35.433273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1027/2056] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:35 2022] PUT /identity/v3/projects/5183d9ba6cb340b4a5c7ce03162c6e64/users/650ab868bb9c4d63b4fff14936ceee0f/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:35.444454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ed32-66cc-463f-a94e-32b7cc611de1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.444738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ed32-66cc-463f-a94e-32b7cc611de1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.445010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ed32-66cc-463f-a94e-32b7cc611de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.472273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:35.475068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f485ed32-66cc-463f-a94e-32b7cc611de1 None None] MFA Rules not processed for user `650ab868bb9c4d63b4fff14936ceee0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:35.476365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f485ed32-66cc-463f-a94e-32b7cc611de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:35.543013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1030/2057] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:35.547431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b665c1-2d6b-486a-97a0-6398063d934b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:35.612260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b665c1-2d6b-486a-97a0-6398063d934b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:35.616999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b6fa3-fdc5-48c7-86bb-71e7615b6b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:35.617597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b6fa3-fdc5-48c7-86bb-71e7615b6b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.617876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b6fa3-fdc5-48c7-86bb-71e7615b6b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.618735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b6fa3-fdc5-48c7-86bb-71e7615b6b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.619440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b6fa3-fdc5-48c7-86bb-71e7615b6b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:35.621341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b6fa3-fdc5-48c7-86bb-71e7615b6b2a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:35.706924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1028/2058] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:35 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:35.711274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ed32-66cc-463f-a94e-32b7cc611de1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:35.721834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ed32-66cc-463f-a94e-32b7cc611de1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:35.777885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fe3fc-6b35-4bce-acf0-6ce69a3cbbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:35.778501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fe3fc-6b35-4bce-acf0-6ce69a3cbbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.778686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fe3fc-6b35-4bce-acf0-6ce69a3cbbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.778983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fe3fc-6b35-4bce-acf0-6ce69a3cbbb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.779611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fe3fc-6b35-4bce-acf0-6ce69a3cbbb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:35.781537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fe3fc-6b35-4bce-acf0-6ce69a3cbbb1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:35.894832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1031/2059] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:35 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:35.898808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b6fa3-fdc5-48c7-86bb-71e7615b6b2a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:35.909064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b6fa3-fdc5-48c7-86bb-71e7615b6b2a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:35.913671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759e5f7-f43f-4faa-943b-655ad3ff4904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:35.914255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e5f7-f43f-4faa-943b-655ad3ff4904 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.914531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e5f7-f43f-4faa-943b-655ad3ff4904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.915372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e5f7-f43f-4faa-943b-655ad3ff4904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.915966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759e5f7-f43f-4faa-943b-655ad3ff4904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:35.917556 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759e5f7-f43f-4faa-943b-655ad3ff4904 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:35.928002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1029/2060] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:35.932640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fe3fc-6b35-4bce-acf0-6ce69a3cbbb1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:35.944575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fe3fc-6b35-4bce-acf0-6ce69a3cbbb1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:35.949214 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2f16a-1965-4921-8231-458d41dcd501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:35.949851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2f16a-1965-4921-8231-458d41dcd501 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:35.950134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2f16a-1965-4921-8231-458d41dcd501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:35.950408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2f16a-1965-4921-8231-458d41dcd501 admin admin] PATH_INFO: `/v3/projects/8352a01036e04d20b9ad1dcdecae41ea/users/e3d077a6615f48a690ca12d806401930/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:35.965709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2f16a-1965-4921-8231-458d41dcd501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8352a01036e04d20b9ad1dcdecae41ea, user_id=e3d077a6615f48a690ca12d806401930, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:35.968777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2f16a-1965-4921-8231-458d41dcd501 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:36.053352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1032/2061] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:35 2022] PUT /identity/v3/projects/8352a01036e04d20b9ad1dcdecae41ea/users/e3d077a6615f48a690ca12d806401930/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:36.057418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759e5f7-f43f-4faa-943b-655ad3ff4904 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:36.067927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759e5f7-f43f-4faa-943b-655ad3ff4904 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:36.107530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:36.125291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8312c32-ec21-4771-a270-a8fe08ae30d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:36.125881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8312c32-ec21-4771-a270-a8fe08ae30d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:36.126160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8312c32-ec21-4771-a270-a8fe08ae30d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:36.126433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8312c32-ec21-4771-a270-a8fe08ae30d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:36.127143 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8312c32-ec21-4771-a270-a8fe08ae30d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:36.128876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8312c32-ec21-4771-a270-a8fe08ae30d1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:36.138986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1030/2062] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:36.143138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2f16a-1965-4921-8231-458d41dcd501 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:36.154411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2f16a-1965-4921-8231-458d41dcd501 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:36.159189 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:36.159916 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:36.160200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:36.160477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] PATH_INFO: `/v3/projects/8352a01036e04d20b9ad1dcdecae41ea/users/e3d077a6615f48a690ca12d806401930/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:36.162389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8352a01036e04d20b9ad1dcdecae41ea, user_id=e3d077a6615f48a690ca12d806401930, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:36.165639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:36.176031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1033/2063] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:36 2022] PUT /identity/v3/projects/8352a01036e04d20b9ad1dcdecae41ea/users/e3d077a6615f48a690ca12d806401930/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:36.186760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69516-5d67-4166-ba8e-3b0adc878b37 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:36.187089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69516-5d67-4166-ba8e-3b0adc878b37 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:36.187393 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69516-5d67-4166-ba8e-3b0adc878b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:36.216161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-80c69516-5d67-4166-ba8e-3b0adc878b37 None None] MFA Rules not processed for user `e3d077a6615f48a690ca12d806401930`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:36.217365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-80c69516-5d67-4166-ba8e-3b0adc878b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:36.284877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1031/2064] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:36.296024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:36.297685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:36.365129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675fd1b-b5a6-4377-9ca8-b7f97ad62b1f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:36.369614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:36.370291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:36.370643 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:36.371043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:36.381128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:36.431183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:36.432642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:36.441906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1034/2065] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:41.966502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69516-5d67-4166-ba8e-3b0adc878b37 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:41.968029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-80c69516-5d67-4166-ba8e-3b0adc878b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:42.048701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:42.050179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:42.066248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69516-5d67-4166-ba8e-3b0adc878b37 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:42.070797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:42.071456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:42.074540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:42.074540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] PATH_INFO: `/v3/users/6dc8bb455f124475b1c053788e05c55c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:42.074540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc8bb455f124475b1c053788e05c55c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:42.075123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:42.133944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836980aa-62cb-410f-aac2-b93c0a30dc2b service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:42.138756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:42.139674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:42.140023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:42.140258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] PATH_INFO: `/v3/users/e3449215c6874ea69749ec369cc508b9` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:42.141431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3449215c6874ea69749ec369cc508b9)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:42.143632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:42.361717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc8bb455f124475b1c053788e05c55c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.362374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc8bb455f124475b1c053788e05c55c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.370903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc8bb455f124475b1c053788e05c55c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.371498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc8bb455f124475b1c053788e05c55c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.382309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3449215c6874ea69749ec369cc508b9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.392313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3449215c6874ea69749ec369cc508b9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.393306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3449215c6874ea69749ec369cc508b9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.393988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3449215c6874ea69749ec369cc508b9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.414143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56afaf68-d2a4-44b2-9cdd-80f9eb85cfde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3449215c6874ea69749ec369cc508b9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.518782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3a6307b1-d3af-41ae-9b5b-7fa8ab1be2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc8bb455f124475b1c053788e05c55c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:42.520249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:42.537839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1032/2066] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:41 2022] DELETE /identity/v3/users/6dc8bb455f124475b1c053788e05c55c => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:42.542155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fee25-4413-4681-a125-d0cc1e754bb8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:42.542439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fee25-4413-4681-a125-d0cc1e754bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:42.542714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fee25-4413-4681-a125-d0cc1e754bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:42.563845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-814fee25-4413-4681-a125-d0cc1e754bb8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:42.565258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-814fee25-4413-4681-a125-d0cc1e754bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:42.590983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1035/2067] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:42 2022] DELETE /identity/v3/users/e3449215c6874ea69749ec369cc508b9 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:42.595149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0001a-8dcf-4663-8b12-20da3e3e9112 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:42.595455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0001a-8dcf-4663-8b12-20da3e3e9112 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:42.595722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0001a-8dcf-4663-8b12-20da3e3e9112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:42.619952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fcd0001a-8dcf-4663-8b12-20da3e3e9112 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:42.621155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fcd0001a-8dcf-4663-8b12-20da3e3e9112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:42.639366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1033/2068] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:42.647747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:42.650971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fee25-4413-4681-a125-d0cc1e754bb8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:42.652400 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-814fee25-4413-4681-a125-d0cc1e754bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:42.690355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1036/2069] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:42.701290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0001a-8dcf-4663-8b12-20da3e3e9112 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:42.703309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fcd0001a-8dcf-4663-8b12-20da3e3e9112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:42.716508 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fee25-4413-4681-a125-d0cc1e754bb8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:42.768333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:42.769009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:42.769282 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:42.769551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:42.780612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:42.782233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:42.793259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1034/2070] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:42.794038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0001a-8dcf-4663-8b12-20da3e3e9112 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:42.798183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:42.798820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:42.799119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:42.799383 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:42.808984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:42.810416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:42.820630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1037/2071] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:43.381666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:43.384567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:43.403319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cc72a-9168-4cff-87ca-758b0a2ad54d service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:43.409702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:43.410587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:43.411031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:43.411164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] PATH_INFO: `/v3/projects/44fa1023082c41ca9cd47378cbf3c191` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:43.412596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44fa1023082c41ca9cd47378cbf3c191)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:43.415313 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:43.454363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:43.455911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:43.490172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad059a74-cc20-4fa0-9fae-1a11ace1872c service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:43.496393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:43.499301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:43.499301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:43.499301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] PATH_INFO: `/v3/projects/b28425f874fb4b00b99a670120c59903` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:43.499301 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28425f874fb4b00b99a670120c59903)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:43.500842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:43.583140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:43.592755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44fa1023082c41ca9cd47378cbf3c191', 'request_id': 'req-a45762a8-1a3b-4dc5-b652-406da67e1301'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.598290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44fa1023082c41ca9cd47378cbf3c191', 'request_id': 'req-a45762a8-1a3b-4dc5-b652-406da67e1301'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.604804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44fa1023082c41ca9cd47378cbf3c191', 'request_id': 'req-a45762a8-1a3b-4dc5-b652-406da67e1301'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.662932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28425f874fb4b00b99a670120c59903', 'request_id': 'req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.733506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] The token cache is being invalidate because project 44fa1023082c41ca9cd47378cbf3c191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:43.733796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.734408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.735927 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1035/2072] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:43 2022] DELETE /identity/v3/projects/44fa1023082c41ca9cd47378cbf3c191 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:43.745089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:43.747944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:43.812343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45762a8-1a3b-4dc5-b652-406da67e1301 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:43.815649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28425f874fb4b00b99a670120c59903', 'request_id': 'req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.822715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28425f874fb4b00b99a670120c59903', 'request_id': 'req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.834394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] The token cache is being invalidate because project b28425f874fb4b00b99a670120c59903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:43.834683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.835303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:43.836781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1038/2073] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:43 2022] DELETE /identity/v3/projects/b28425f874fb4b00b99a670120c59903 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:43.840825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:43.842375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:43.870644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:43.871308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:43.871589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:43.871864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] PATH_INFO: `/v3/users/e9e4c649ec5d418486aec2cd44ac4c41` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:43.873009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9e4c649ec5d418486aec2cd44ac4c41)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:43.874964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:43.902631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:43.907894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0aa9e-26de-47f5-ab7c-cf5fa336301e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:43.912615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:43.913299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:43.913525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:43.913798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] PATH_INFO: `/v3/users/90de5ccbf073420ebc29e25205e98659` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:43.915006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90de5ccbf073420ebc29e25205e98659)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:43.917551 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:44.069739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9e4c649ec5d418486aec2cd44ac4c41'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.070427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9e4c649ec5d418486aec2cd44ac4c41'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.077849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9e4c649ec5d418486aec2cd44ac4c41'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.078501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9e4c649ec5d418486aec2cd44ac4c41'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.193219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9e4c649ec5d418486aec2cd44ac4c41'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.214458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1036/2074] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:43 2022] DELETE /identity/v3/users/e9e4c649ec5d418486aec2cd44ac4c41 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:44.231368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90de5ccbf073420ebc29e25205e98659'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.239096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90de5ccbf073420ebc29e25205e98659'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.239890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90de5ccbf073420ebc29e25205e98659'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.245335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90de5ccbf073420ebc29e25205e98659'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.263974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90de5ccbf073420ebc29e25205e98659'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:44.355344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1039/2075] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:43 2022] DELETE /identity/v3/users/90de5ccbf073420ebc29e25205e98659 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:44.423179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:44.732398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:44.732627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:44.823546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad19f7-5aff-4925-94a3-bd0333921339 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:44.889617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:44.890372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:44.890655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:44.890928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] PATH_INFO: `/v3/projects/fe9bac65b3a0465db5336b698d88bebe` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:44.892100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9bac65b3a0465db5336b698d88bebe)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:44.894009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:45.055795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:45.057996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:45.064312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9bac65b3a0465db5336b698d88bebe', 'request_id': 'req-73024eab-82e6-487b-aaa4-3cfd65054700'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.071432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9bac65b3a0465db5336b698d88bebe', 'request_id': 'req-73024eab-82e6-487b-aaa4-3cfd65054700'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.079842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9bac65b3a0465db5336b698d88bebe', 'request_id': 'req-73024eab-82e6-487b-aaa4-3cfd65054700'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.080017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c7716-cd03-4cd4-92bc-a52477260d15 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:45.137987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:45.138607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:45.138881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:45.139177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] PATH_INFO: `/v3/projects/6389dab63b984b8181cf0b35aa30db62` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:45.140266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6389dab63b984b8181cf0b35aa30db62)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:45.142158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:45.200221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] The token cache is being invalidate because project fe9bac65b3a0465db5336b698d88bebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:45.202623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.202623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.203152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1037/2076] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:44 2022] DELETE /identity/v3/projects/fe9bac65b3a0465db5336b698d88bebe => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:45.206472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:45.208134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:45.220904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73024eab-82e6-487b-aaa4-3cfd65054700 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:45.225615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:45.226250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:45.226522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:45.226803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] PATH_INFO: `/v3/users/ba76f05db5094413989c32dbc323345c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:45.227959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba76f05db5094413989c32dbc323345c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:45.229902 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:45.252381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:45.290576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6389dab63b984b8181cf0b35aa30db62', 'request_id': 'req-4da2997c-bc18-41e8-add1-d6fd1da417a4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.393477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6389dab63b984b8181cf0b35aa30db62', 'request_id': 'req-4da2997c-bc18-41e8-add1-d6fd1da417a4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.403432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6389dab63b984b8181cf0b35aa30db62', 'request_id': 'req-4da2997c-bc18-41e8-add1-d6fd1da417a4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.413993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] The token cache is being invalidate because project 6389dab63b984b8181cf0b35aa30db62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:45.414290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.414932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4da2997c-bc18-41e8-add1-d6fd1da417a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.416966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1040/2077] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:45 2022] DELETE /identity/v3/projects/6389dab63b984b8181cf0b35aa30db62 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:45.421772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32930026-1700-4fba-b953-d5dc6d602667 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:45.422060 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32930026-1700-4fba-b953-d5dc6d602667 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:45.422336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32930026-1700-4fba-b953-d5dc6d602667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:45.430849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:45.442002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-32930026-1700-4fba-b953-d5dc6d602667 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:45.443238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-32930026-1700-4fba-b953-d5dc6d602667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:45.500770 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1041/2078] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:45.509422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9dd81-0b3e-4496-b65d-a12406fe7eec None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:45.509710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9dd81-0b3e-4496-b65d-a12406fe7eec None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:45.509983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9dd81-0b3e-4496-b65d-a12406fe7eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:45.528633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-41b9dd81-0b3e-4496-b65d-a12406fe7eec None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:45.529836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-41b9dd81-0b3e-4496-b65d-a12406fe7eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:45.538012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1042/2079] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:45.543467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9dd81-0b3e-4496-b65d-a12406fe7eec None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:45.545190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba76f05db5094413989c32dbc323345c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.545877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba76f05db5094413989c32dbc323345c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.552810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9dd81-0b3e-4496-b65d-a12406fe7eec None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:45.553030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba76f05db5094413989c32dbc323345c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.553746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba76f05db5094413989c32dbc323345c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.558151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:45.558738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:45.559047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:45.559344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:45.559987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:45.560284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:45.561873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:45.573309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1043/2080] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:18:45.577481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:45.579158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:45.591790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd902f-2e52-4fa3-b938-74c5d4b3dd74 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:45.596427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cc909-0964-428d-8d04-1cf52a7a3d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:45.596987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cc909-0964-428d-8d04-1cf52a7a3d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:45.597266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cc909-0964-428d-8d04-1cf52a7a3d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:45.597539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cc909-0964-428d-8d04-1cf52a7a3d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:45.598151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cc909-0964-428d-8d04-1cf52a7a3d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:45.600190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cc909-0964-428d-8d04-1cf52a7a3d1f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:45.667870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba76f05db5094413989c32dbc323345c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:45.685922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1038/2081] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:45 2022] DELETE /identity/v3/users/ba76f05db5094413989c32dbc323345c => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:45.732440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1044/2082] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:45 2022] POST /identity/v3/projects => generated 409 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:45.736756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:45.804198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef48c6-3318-4e95-9812-cc1f0d3893e0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:45.861211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bb6ed-41a5-489c-b8ed-5076694cca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:45.861820 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb6ed-41a5-489c-b8ed-5076694cca32 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:45.862106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb6ed-41a5-489c-b8ed-5076694cca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:45.862378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bb6ed-41a5-489c-b8ed-5076694cca32 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:45.863049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bb6ed-41a5-489c-b8ed-5076694cca32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:45.865061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bb6ed-41a5-489c-b8ed-5076694cca32 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:45.989107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1039/2083] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:45 2022] POST /identity/v3/users => generated 305 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:45.994234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cc909-0964-428d-8d04-1cf52a7a3d1f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.017761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cc909-0964-428d-8d04-1cf52a7a3d1f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.022042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291819e0-3258-4403-b898-fd5f129c35b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.022718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291819e0-3258-4403-b898-fd5f129c35b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.023326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291819e0-3258-4403-b898-fd5f129c35b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.023488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291819e0-3258-4403-b898-fd5f129c35b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.024205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291819e0-3258-4403-b898-fd5f129c35b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.025955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291819e0-3258-4403-b898-fd5f129c35b2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.040870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1045/2084] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:46.042218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bb6ed-41a5-489c-b8ed-5076694cca32 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.053957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bb6ed-41a5-489c-b8ed-5076694cca32 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.072133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.072761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.073043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.073321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.074115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.076342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.088560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1040/2085] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:46.092753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291819e0-3258-4403-b898-fd5f129c35b2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.103028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291819e0-3258-4403-b898-fd5f129c35b2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.108414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5a74d-45a8-4925-983b-17268a1e7159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.118034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5a74d-45a8-4925-983b-17268a1e7159 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.118034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5a74d-45a8-4925-983b-17268a1e7159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.118034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5a74d-45a8-4925-983b-17268a1e7159 admin admin] PATH_INFO: `/v3/projects/b56c6fc64aef466baaa3981132efe579/users/7a0b9b0492634d1a90819d5086d59378/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.125577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5a74d-45a8-4925-983b-17268a1e7159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56c6fc64aef466baaa3981132efe579, user_id=7a0b9b0492634d1a90819d5086d59378, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.128284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5a74d-45a8-4925-983b-17268a1e7159 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.372354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1046/2086] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:46 2022] PUT /identity/v3/projects/b56c6fc64aef466baaa3981132efe579/users/7a0b9b0492634d1a90819d5086d59378/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:46.375110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.375271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:46.387718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933be91-1386-482f-98d3-0a6305e77d40 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.387874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933be91-1386-482f-98d3-0a6305e77d40 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.388135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933be91-1386-482f-98d3-0a6305e77d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.389483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b6130-42bc-48d0-8d1a-0f245df51f55 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.414842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:46.417320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e933be91-1386-482f-98d3-0a6305e77d40 None None] MFA Rules not processed for user `7a0b9b0492634d1a90819d5086d59378`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:46.418538 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e933be91-1386-482f-98d3-0a6305e77d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:46.453083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.455479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.455479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.455479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] PATH_INFO: `/v3/projects/4bd2f1ea35164d4399baae86a56bb45e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.455830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd2f1ea35164d4399baae86a56bb45e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.457585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.488327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1047/2087] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:46 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:46.492776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933be91-1386-482f-98d3-0a6305e77d40 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.503229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933be91-1386-482f-98d3-0a6305e77d40 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.508242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aac98-9b6d-48f3-b444-f1103ab0209c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.511609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aac98-9b6d-48f3-b444-f1103ab0209c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.511609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aac98-9b6d-48f3-b444-f1103ab0209c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.511609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aac98-9b6d-48f3-b444-f1103ab0209c admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.511609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aac98-9b6d-48f3-b444-f1103ab0209c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.512407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aac98-9b6d-48f3-b444-f1103ab0209c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.529843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:46.599709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd2f1ea35164d4399baae86a56bb45e', 'request_id': 'req-dc670532-b30f-43af-905c-01890ca53235'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:46.605247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd2f1ea35164d4399baae86a56bb45e', 'request_id': 'req-dc670532-b30f-43af-905c-01890ca53235'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:46.611819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1048/2088] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:46 2022] POST /identity/v3/projects => generated 407 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:46.615061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd2f1ea35164d4399baae86a56bb45e', 'request_id': 'req-dc670532-b30f-43af-905c-01890ca53235'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:46.615944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aac98-9b6d-48f3-b444-f1103ab0209c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.677649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aac98-9b6d-48f3-b444-f1103ab0209c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.683193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.683801 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.684070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.684343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.684961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.686911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.713169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] The token cache is being invalidate because project 4bd2f1ea35164d4399baae86a56bb45e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:46.713462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:46.714089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc670532-b30f-43af-905c-01890ca53235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:46.715622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1041/2089] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:46 2022] DELETE /identity/v3/projects/4bd2f1ea35164d4399baae86a56bb45e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:46.720336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dbfd6-f419-4231-989d-a281954e068e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.720625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dbfd6-f419-4231-989d-a281954e068e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.720903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dbfd6-f419-4231-989d-a281954e068e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.739804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-839dbfd6-f419-4231-989d-a281954e068e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:46.741009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-839dbfd6-f419-4231-989d-a281954e068e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:46.749585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1042/2090] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:46.757009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2016a-19a1-490a-8098-4d0bf1f4696b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.757294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2016a-19a1-490a-8098-4d0bf1f4696b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.757571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2016a-19a1-490a-8098-4d0bf1f4696b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.777062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-85b2016a-19a1-490a-8098-4d0bf1f4696b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:46.778255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-85b2016a-19a1-490a-8098-4d0bf1f4696b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:46.786406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1043/2091] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:46.790486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2016a-19a1-490a-8098-4d0bf1f4696b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.799751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2016a-19a1-490a-8098-4d0bf1f4696b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.804478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.813734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.813868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1049/2092] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:46 2022] POST /identity/v3/users => generated 310 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:46.813972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.813972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.813972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:46.813972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.813972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.816354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.818013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:46.824130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1044/2093] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:18:46.825386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.831117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be061b-339f-45d0-a939-b5b2d6963ed8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.835517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541fc14-6fbd-4e01-a71e-7fea203565a0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.836311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52108a-cb00-4ad4-a970-63a8ab26a5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.837091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52108a-cb00-4ad4-a970-63a8ab26a5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.837908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52108a-cb00-4ad4-a970-63a8ab26a5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.838142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52108a-cb00-4ad4-a970-63a8ab26a5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.838868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52108a-cb00-4ad4-a970-63a8ab26a5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.840415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7818a-c3ad-483f-a990-2ad3070a90d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.843127 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52108a-cb00-4ad4-a970-63a8ab26a5a4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.843254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7818a-c3ad-483f-a990-2ad3070a90d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.843254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7818a-c3ad-483f-a990-2ad3070a90d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.843254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7818a-c3ad-483f-a990-2ad3070a90d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.843254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7818a-c3ad-483f-a990-2ad3070a90d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.844001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7818a-c3ad-483f-a990-2ad3070a90d9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.850766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1050/2094] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:46.855387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52108a-cb00-4ad4-a970-63a8ab26a5a4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.864900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52108a-cb00-4ad4-a970-63a8ab26a5a4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:46.870579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb1eb1-9cd0-4efe-8b9a-c87f463d77ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:46.871268 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb1eb1-9cd0-4efe-8b9a-c87f463d77ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:46.871502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb1eb1-9cd0-4efe-8b9a-c87f463d77ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:46.876833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb1eb1-9cd0-4efe-8b9a-c87f463d77ab admin admin] PATH_INFO: `/v3/projects/ab00b264001746b0b535282adcff618c/users/80d977b6686542f4adf87b0006cfbf8a/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:46.886373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb1eb1-9cd0-4efe-8b9a-c87f463d77ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab00b264001746b0b535282adcff618c, user_id=80d977b6686542f4adf87b0006cfbf8a, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:46.889330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb1eb1-9cd0-4efe-8b9a-c87f463d77ab admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:46.938117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1045/2095] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:46.942399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7818a-c3ad-483f-a990-2ad3070a90d9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:46.962946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1051/2096] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:46 2022] PUT /identity/v3/projects/ab00b264001746b0b535282adcff618c/users/80d977b6686542f4adf87b0006cfbf8a/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:46.969298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb1eb1-9cd0-4efe-8b9a-c87f463d77ab admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:47.009876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7818a-c3ad-483f-a990-2ad3070a90d9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.023202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:47.047241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb1eb1-9cd0-4efe-8b9a-c87f463d77ab admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.052728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ec2df-0ead-41ac-9c63-d11027843e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.053490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ec2df-0ead-41ac-9c63-d11027843e52 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.053768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ec2df-0ead-41ac-9c63-d11027843e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.054110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ec2df-0ead-41ac-9c63-d11027843e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.054971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ec2df-0ead-41ac-9c63-d11027843e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.057473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ec2df-0ead-41ac-9c63-d11027843e52 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.074986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1052/2097] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:47.081290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ec2df-0ead-41ac-9c63-d11027843e52 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:47.082340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b8fa7-c027-4971-b2fc-5541247d9570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.083304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b8fa7-c027-4971-b2fc-5541247d9570 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.083683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b8fa7-c027-4971-b2fc-5541247d9570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.084302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b8fa7-c027-4971-b2fc-5541247d9570 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.085283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b8fa7-c027-4971-b2fc-5541247d9570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.087831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b8fa7-c027-4971-b2fc-5541247d9570 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.092904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ec2df-0ead-41ac-9c63-d11027843e52 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.098179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08797771-0117-4b87-ac70-85ea141e45e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.099047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08797771-0117-4b87-ac70-85ea141e45e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.099395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08797771-0117-4b87-ac70-85ea141e45e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.099804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08797771-0117-4b87-ac70-85ea141e45e2 admin admin] PATH_INFO: `/v3/projects/ab00b264001746b0b535282adcff618c/users/80d977b6686542f4adf87b0006cfbf8a/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.101957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08797771-0117-4b87-ac70-85ea141e45e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab00b264001746b0b535282adcff618c, user_id=80d977b6686542f4adf87b0006cfbf8a, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.105278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08797771-0117-4b87-ac70-85ea141e45e2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.123242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1053/2098] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:47 2022] PUT /identity/v3/projects/ab00b264001746b0b535282adcff618c/users/80d977b6686542f4adf87b0006cfbf8a/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:47.128015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17af6f3-6a92-4d38-9f2c-2ae9e30efd4b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.128306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17af6f3-6a92-4d38-9f2c-2ae9e30efd4b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.128584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17af6f3-6a92-4d38-9f2c-2ae9e30efd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.157817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d17af6f3-6a92-4d38-9f2c-2ae9e30efd4b None None] MFA Rules not processed for user `80d977b6686542f4adf87b0006cfbf8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:47.159058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d17af6f3-6a92-4d38-9f2c-2ae9e30efd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:47.180568 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:47.212205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1046/2099] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:46 2022] POST /identity/v3/users => generated 287 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:18:47.217801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b8fa7-c027-4971-b2fc-5541247d9570 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:47.229074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b8fa7-c027-4971-b2fc-5541247d9570 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.242247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1054/2100] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:47 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:47.253281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17af6f3-6a92-4d38-9f2c-2ae9e30efd4b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:47.254795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d17af6f3-6a92-4d38-9f2c-2ae9e30efd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:47.297794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84511bd-7bac-4bcc-bea1-9f8d140ef9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.298402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84511bd-7bac-4bcc-bea1-9f8d140ef9da admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.298678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84511bd-7bac-4bcc-bea1-9f8d140ef9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.298988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84511bd-7bac-4bcc-bea1-9f8d140ef9da admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.299630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84511bd-7bac-4bcc-bea1-9f8d140ef9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.301253 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84511bd-7bac-4bcc-bea1-9f8d140ef9da admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.311658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1047/2101] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:47.315778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84511bd-7bac-4bcc-bea1-9f8d140ef9da admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:47.316660 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17af6f3-6a92-4d38-9f2c-2ae9e30efd4b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.320299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b6f92-984d-40fd-8764-3919fbd20824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.320904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b6f92-984d-40fd-8764-3919fbd20824 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.321178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b6f92-984d-40fd-8764-3919fbd20824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.321451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b6f92-984d-40fd-8764-3919fbd20824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.325897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84511bd-7bac-4bcc-bea1-9f8d140ef9da admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.330582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81448eb9-fa38-4d98-aeb8-f4c17125fb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.331174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81448eb9-fa38-4d98-aeb8-f4c17125fb21 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.331450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81448eb9-fa38-4d98-aeb8-f4c17125fb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.331719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81448eb9-fa38-4d98-aeb8-f4c17125fb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.332008 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b6f92-984d-40fd-8764-3919fbd20824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.332396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81448eb9-fa38-4d98-aeb8-f4c17125fb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.333478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b6f92-984d-40fd-8764-3919fbd20824 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.333950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81448eb9-fa38-4d98-aeb8-f4c17125fb21 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.343689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1055/2102] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:47 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:18:47.345068 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1048/2103] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:18:47.349173 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b6f92-984d-40fd-8764-3919fbd20824 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:47.359363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b6f92-984d-40fd-8764-3919fbd20824 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.363962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.364593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.364868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.365137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] PATH_INFO: `/v3/projects/cdbd04906d2b42fba52e61f3dbd0af21/users/45777028be184247b0a58eee7d2c7e45/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.380523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbd04906d2b42fba52e61f3dbd0af21, user_id=45777028be184247b0a58eee7d2c7e45, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.383195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.479586 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1056/2104] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:47 2022] PUT /identity/v3/projects/cdbd04906d2b42fba52e61f3dbd0af21/users/45777028be184247b0a58eee7d2c7e45/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:18:47.484772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6bf4-db9a-45a4-b119-58544c31e47c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.491583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6bf4-db9a-45a4-b119-58544c31e47c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.491583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6bf4-db9a-45a4-b119-58544c31e47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.504420 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-641b6bf4-db9a-45a4-b119-58544c31e47c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:47.505624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-641b6bf4-db9a-45a4-b119-58544c31e47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:47.564988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1049/2105] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:47.575266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:47.576984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:47.634110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24141e9-a4f5-4b63-b16a-7bfd6325e200 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.637820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162b5ce-48e6-40af-a2f2-ce550985a3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.638421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b5ce-48e6-40af-a2f2-ce550985a3dd service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.638697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b5ce-48e6-40af-a2f2-ce550985a3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.638991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b5ce-48e6-40af-a2f2-ce550985a3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.647853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162b5ce-48e6-40af-a2f2-ce550985a3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.649277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162b5ce-48e6-40af-a2f2-ce550985a3dd service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.658560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1057/2106] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:47.750860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b6bf4-db9a-45a4-b119-58544c31e47c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:47.782027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:47.807969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b6bf4-db9a-45a4-b119-58544c31e47c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:47.815921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:47.815921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:47.815921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:47.815921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:47.816252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:47.875249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:47.876692 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:47.885751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1050/2107] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:47 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:49.399054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adafe3-0cb9-472f-b33d-006f0c078670 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:49.399054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adafe3-0cb9-472f-b33d-006f0c078670 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:49.399054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adafe3-0cb9-472f-b33d-006f0c078670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:49.432289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-03adafe3-0cb9-472f-b33d-006f0c078670 None None] MFA Rules not processed for user `45777028be184247b0a58eee7d2c7e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:49.433678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-03adafe3-0cb9-472f-b33d-006f0c078670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:49.473558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:49.520459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1058/2108] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:49.532169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:49.536283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:49.547149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0deb5-57c7-486f-95ef-85d9953a56c9 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:49.550893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7f704-a6b4-4be5-99d7-0087bae7ae86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:49.551527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7f704-a6b4-4be5-99d7-0087bae7ae86 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:49.551801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7f704-a6b4-4be5-99d7-0087bae7ae86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:49.552066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7f704-a6b4-4be5-99d7-0087bae7ae86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:49.561306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7f704-a6b4-4be5-99d7-0087bae7ae86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:49.562728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7f704-a6b4-4be5-99d7-0087bae7ae86 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:49.573273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1051/2109] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:50.436252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adafe3-0cb9-472f-b33d-006f0c078670 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:50.446387 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adafe3-0cb9-472f-b33d-006f0c078670 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:50.450385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f55bd-bcef-43d7-b410-adee3f04e61d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:50.451042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f55bd-bcef-43d7-b410-adee3f04e61d service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:50.451322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f55bd-bcef-43d7-b410-adee3f04e61d service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:50.451632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f55bd-bcef-43d7-b410-adee3f04e61d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:50.453610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a39f55bd-bcef-43d7-b410-adee3f04e61d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:50.519332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f55bd-bcef-43d7-b410-adee3f04e61d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:50.520784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f55bd-bcef-43d7-b410-adee3f04e61d service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:50.530910 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1059/2110] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:55.424337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3701dff-4522-49f0-b203-cdf8b2a1c9d8 None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:55.425210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3701dff-4522-49f0-b203-cdf8b2a1c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:55.425642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3701dff-4522-49f0-b203-cdf8b2a1c9d8 None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:55.427335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1052/2111] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:18:55.432573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7f87-2530-427a-baeb-c67203457baf None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:55.433180 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7f87-2530-427a-baeb-c67203457baf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:55.433939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7f87-2530-427a-baeb-c67203457baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:55.456692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3c0d7f87-2530-427a-baeb-c67203457baf None None] MFA Rules not processed for user `40df52dd29044cddbd521b59dd30b6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:55.457907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c0d7f87-2530-427a-baeb-c67203457baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:55.526769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1060/2112] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:18:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:55.537314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3701dff-4522-49f0-b203-cdf8b2a1c9d8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:55.538960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d3701dff-4522-49f0-b203-cdf8b2a1c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:55.555352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3701dff-4522-49f0-b203-cdf8b2a1c9d8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:55.563783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596af6b2-2f37-4d5d-a354-de5fc353dd63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:55.563783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596af6b2-2f37-4d5d-a354-de5fc353dd63 service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:55.563783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596af6b2-2f37-4d5d-a354-de5fc353dd63 service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:55.563783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596af6b2-2f37-4d5d-a354-de5fc353dd63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:55.572409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596af6b2-2f37-4d5d-a354-de5fc353dd63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:55.574720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596af6b2-2f37-4d5d-a354-de5fc353dd63 service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:55.586751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1053/2113] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:18:55.604018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7f87-2530-427a-baeb-c67203457baf None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:55.605657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c0d7f87-2530-427a-baeb-c67203457baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:55.621113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7f87-2530-427a-baeb-c67203457baf None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:55.626177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:55.626875 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:55.627473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:55.627616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] PATH_INFO: `/v3/users/dd81f4241d7343e990aea15e49be5f81` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:55.628781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd81f4241d7343e990aea15e49be5f81)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:55.630696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:55.875682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd81f4241d7343e990aea15e49be5f81'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:55.882449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:55.884922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd81f4241d7343e990aea15e49be5f81'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:55.885775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd81f4241d7343e990aea15e49be5f81'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:55.886695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd81f4241d7343e990aea15e49be5f81'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:55.900909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fa7df61e-6a7e-4dcc-b37e-309384f92fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd81f4241d7343e990aea15e49be5f81'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:55.971442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1061/2114] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:18:55 2022] DELETE /identity/v3/users/dd81f4241d7343e990aea15e49be5f81 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:56.219601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596af6b2-2f37-4d5d-a354-de5fc353dd63 service placement] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:56.233897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596af6b2-2f37-4d5d-a354-de5fc353dd63 service placement] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:56.290581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:56.291239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:56.291566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:56.292360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] PATH_INFO: `/v3/projects/7d310b6c64154e3bb5d95f1e762e3e80` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:56.293485 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d310b6c64154e3bb5d95f1e762e3e80)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:56.295483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:56.438392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d310b6c64154e3bb5d95f1e762e3e80', 'request_id': 'req-367b195d-dd92-47c4-ba12-08af4a43fd8a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:56.443931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d310b6c64154e3bb5d95f1e762e3e80', 'request_id': 'req-367b195d-dd92-47c4-ba12-08af4a43fd8a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:56.445150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:56.450627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d310b6c64154e3bb5d95f1e762e3e80', 'request_id': 'req-367b195d-dd92-47c4-ba12-08af4a43fd8a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:56.597932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] The token cache is being invalidate because project 7d310b6c64154e3bb5d95f1e762e3e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:18:56.598175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:56.598748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-367b195d-dd92-47c4-ba12-08af4a43fd8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:18:56.600388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1054/2115] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:18:56 2022] DELETE /identity/v3/projects/7d310b6c64154e3bb5d95f1e762e3e80 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:56.606048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95d187-b5fd-4ff2-bd90-cfc788027368 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:56.606525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95d187-b5fd-4ff2-bd90-cfc788027368 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:56.612142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95d187-b5fd-4ff2-bd90-cfc788027368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:56.631161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3e95d187-b5fd-4ff2-bd90-cfc788027368 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:56.632528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3e95d187-b5fd-4ff2-bd90-cfc788027368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:56.716077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1062/2116] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:56.724098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cca9dc-dc7c-42f7-8891-c5505ad3aa17 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:56.724499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cca9dc-dc7c-42f7-8891-c5505ad3aa17 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:56.724703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cca9dc-dc7c-42f7-8891-c5505ad3aa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:56.744818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-57cca9dc-dc7c-42f7-8891-c5505ad3aa17 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:56.746123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-57cca9dc-dc7c-42f7-8891-c5505ad3aa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:56.754694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1055/2117] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:56.758569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95d187-b5fd-4ff2-bd90-cfc788027368 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:56.769068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95d187-b5fd-4ff2-bd90-cfc788027368 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:56.773766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:56.774362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:56.774639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:56.774913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:56.775591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:18:56.776548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:56.778160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:56.790574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1063/2118] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:18:56.795001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cca9dc-dc7c-42f7-8891-c5505ad3aa17 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:56.804386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cca9dc-dc7c-42f7-8891-c5505ad3aa17 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:56.809040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd6cb-cd9e-442c-8fa7-60412be03286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:56.809637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd6cb-cd9e-442c-8fa7-60412be03286 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:56.809919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd6cb-cd9e-442c-8fa7-60412be03286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:56.810189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd6cb-cd9e-442c-8fa7-60412be03286 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:56.810816 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fd6cb-cd9e-442c-8fa7-60412be03286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:56.812739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fd6cb-cd9e-442c-8fa7-60412be03286 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:56.944877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1056/2119] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:56 2022] POST /identity/v3/projects => generated 403 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:56.950177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:56.960352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b0325-4d49-44e6-9859-bf4647e4dd17 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:57.018373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d48b-f25b-4af2-b501-1d54b6257519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:57.019042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d48b-f25b-4af2-b501-1d54b6257519 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.019354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d48b-f25b-4af2-b501-1d54b6257519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.019626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0d48b-f25b-4af2-b501-1d54b6257519 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.020282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0d48b-f25b-4af2-b501-1d54b6257519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:57.022211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0d48b-f25b-4af2-b501-1d54b6257519 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:57.124376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1064/2120] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:56 2022] POST /identity/v3/users => generated 302 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:57.128388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd6cb-cd9e-442c-8fa7-60412be03286 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:57.139604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd6cb-cd9e-442c-8fa7-60412be03286 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:57.144250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d73de-29ac-4abc-be43-afcd18d63f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:57.144872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d73de-29ac-4abc-be43-afcd18d63f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.145147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d73de-29ac-4abc-be43-afcd18d63f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.145441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d73de-29ac-4abc-be43-afcd18d63f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.146044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d73de-29ac-4abc-be43-afcd18d63f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:57.147674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d73de-29ac-4abc-be43-afcd18d63f09 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:57.157431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1057/2121] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:57.161366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d48b-f25b-4af2-b501-1d54b6257519 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:57.171448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0d48b-f25b-4af2-b501-1d54b6257519 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:57.176512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0904-6080-4eb7-8ae9-6127c5d1754c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:57.177112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0904-6080-4eb7-8ae9-6127c5d1754c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.177423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0904-6080-4eb7-8ae9-6127c5d1754c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.177697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0904-6080-4eb7-8ae9-6127c5d1754c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.178339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0904-6080-4eb7-8ae9-6127c5d1754c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:57.180063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0904-6080-4eb7-8ae9-6127c5d1754c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:57.191497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1065/2122] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:57.195753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d73de-29ac-4abc-be43-afcd18d63f09 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:57.206653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d73de-29ac-4abc-be43-afcd18d63f09 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:57.211713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7327e-4082-4fe6-99a5-a24992d7f9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:57.212358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7327e-4082-4fe6-99a5-a24992d7f9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.212636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7327e-4082-4fe6-99a5-a24992d7f9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.212911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7327e-4082-4fe6-99a5-a24992d7f9b5 admin admin] PATH_INFO: `/v3/projects/14d74e54e8fb44d4a97a561fd818c519/users/aa26c019cb704ddda51493c411e57a0a/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.227516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7327e-4082-4fe6-99a5-a24992d7f9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d74e54e8fb44d4a97a561fd818c519, user_id=aa26c019cb704ddda51493c411e57a0a, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:57.230236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7327e-4082-4fe6-99a5-a24992d7f9b5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:57.343743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1058/2123] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:57 2022] PUT /identity/v3/projects/14d74e54e8fb44d4a97a561fd818c519/users/aa26c019cb704ddda51493c411e57a0a/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:57.354651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c730f1-b1fd-495f-9852-ee4ef3632cc7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.354959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c730f1-b1fd-495f-9852-ee4ef3632cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.355238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c730f1-b1fd-495f-9852-ee4ef3632cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.384195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-82c730f1-b1fd-495f-9852-ee4ef3632cc7 None None] MFA Rules not processed for user `aa26c019cb704ddda51493c411e57a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:57.385411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-82c730f1-b1fd-495f-9852-ee4ef3632cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:57.412636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:57.453267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1066/2124] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:57 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:57.458336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7327e-4082-4fe6-99a5-a24992d7f9b5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:57.525924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7327e-4082-4fe6-99a5-a24992d7f9b5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:57.530669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582e681-6068-448f-b922-1b966b3b6d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:57.531307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e681-6068-448f-b922-1b966b3b6d29 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.531590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e681-6068-448f-b922-1b966b3b6d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.531864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e681-6068-448f-b922-1b966b3b6d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.532496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582e681-6068-448f-b922-1b966b3b6d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:57.534388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582e681-6068-448f-b922-1b966b3b6d29 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:57.641609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1059/2125] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:18:57 2022] POST /identity/v3/projects => generated 405 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:57.645769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c730f1-b1fd-495f-9852-ee4ef3632cc7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:57.656722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c730f1-b1fd-495f-9852-ee4ef3632cc7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:57.739820 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373623-1208-4a01-92ab-1d926ac190f5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.740281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373623-1208-4a01-92ab-1d926ac190f5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.741755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07373623-1208-4a01-92ab-1d926ac190f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.782814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-07373623-1208-4a01-92ab-1d926ac190f5 None None] MFA Rules not processed for user `45777028be184247b0a58eee7d2c7e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:57.784604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150db28-aa02-4868-b342-549fd0263e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:57.785470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150db28-aa02-4868-b342-549fd0263e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.785749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150db28-aa02-4868-b342-549fd0263e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.786523 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150db28-aa02-4868-b342-549fd0263e48 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.786612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-07373623-1208-4a01-92ab-1d926ac190f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:57.786688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150db28-aa02-4868-b342-549fd0263e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:57.788700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150db28-aa02-4868-b342-549fd0263e48 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:57.856718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1060/2126] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:18:57.892254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1067/2127] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:18:57 2022] POST /identity/v3/users => generated 309 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:18:57.895415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07373623-1208-4a01-92ab-1d926ac190f5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:57.910754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07373623-1208-4a01-92ab-1d926ac190f5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:57.915557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d4e28-f327-42a4-bc32-ec94142d58da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:57.917237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d4e28-f327-42a4-bc32-ec94142d58da admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.917655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d4e28-f327-42a4-bc32-ec94142d58da admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.917775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d4e28-f327-42a4-bc32-ec94142d58da admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.918499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d4e28-f327-42a4-bc32-ec94142d58da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:57.920159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d4e28-f327-42a4-bc32-ec94142d58da admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:57.931123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1061/2128] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:57.935405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150db28-aa02-4868-b342-549fd0263e48 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:57.948032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150db28-aa02-4868-b342-549fd0263e48 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:57.953018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eff6a-9b60-430f-94b7-383425955bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:57.953808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eff6a-9b60-430f-94b7-383425955bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:57.954172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eff6a-9b60-430f-94b7-383425955bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:57.954564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eff6a-9b60-430f-94b7-383425955bfe admin admin] PATH_INFO: `/v3/projects/d384e07649e042529d5b3391e1505b36/users/43388fad959a40fd9c94670fc58baf0b/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:57.976883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88eff6a-9b60-430f-94b7-383425955bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d384e07649e042529d5b3391e1505b36, user_id=43388fad959a40fd9c94670fc58baf0b, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:57.980269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88eff6a-9b60-430f-94b7-383425955bfe admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:58.046832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1068/2129] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:57 2022] PUT /identity/v3/projects/d384e07649e042529d5b3391e1505b36/users/43388fad959a40fd9c94670fc58baf0b/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:58.050696 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d4e28-f327-42a4-bc32-ec94142d58da admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:58.056152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:18:58.061676 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d4e28-f327-42a4-bc32-ec94142d58da admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:58.126498 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f19b3-f1bf-4ea3-a53d-ddaf2795f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:58.127133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f19b3-f1bf-4ea3-a53d-ddaf2795f0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:58.127409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f19b3-f1bf-4ea3-a53d-ddaf2795f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:58.127730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f19b3-f1bf-4ea3-a53d-ddaf2795f0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:58.128318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f19b3-f1bf-4ea3-a53d-ddaf2795f0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:58.129965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f19b3-f1bf-4ea3-a53d-ddaf2795f0b4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:58.140471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1062/2130] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:18:58.144817 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eff6a-9b60-430f-94b7-383425955bfe admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:58.155646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eff6a-9b60-430f-94b7-383425955bfe admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:58.160277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:58.160918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:58.161196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:58.161466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] PATH_INFO: `/v3/projects/d384e07649e042529d5b3391e1505b36/users/43388fad959a40fd9c94670fc58baf0b/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:58.163383 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d384e07649e042529d5b3391e1505b36, user_id=43388fad959a40fd9c94670fc58baf0b, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:58.166049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:58.176496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1069/2131] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:18:58 2022] PUT /identity/v3/projects/d384e07649e042529d5b3391e1505b36/users/43388fad959a40fd9c94670fc58baf0b/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:18:58.188279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c4bc-6bb3-4a1a-8f86-de8359f9776a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:58.188491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c4bc-6bb3-4a1a-8f86-de8359f9776a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:58.188762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c4bc-6bb3-4a1a-8f86-de8359f9776a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:58.221469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a481c4bc-6bb3-4a1a-8f86-de8359f9776a None None] MFA Rules not processed for user `43388fad959a40fd9c94670fc58baf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:18:58.222694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a481c4bc-6bb3-4a1a-8f86-de8359f9776a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:58.300528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1063/2132] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:18:58 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:18:58.311886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:18:58.313683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:18:58.374561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c03a3-ddb1-4d64-8fd3-53fbdd436285 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:18:58.378215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77656bb7-0214-4b05-b552-66a8a5cf033d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:18:58.378815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77656bb7-0214-4b05-b552-66a8a5cf033d service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:18:58.379136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77656bb7-0214-4b05-b552-66a8a5cf033d service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:18:58.379413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77656bb7-0214-4b05-b552-66a8a5cf033d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:18:58.388381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77656bb7-0214-4b05-b552-66a8a5cf033d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:18:58.389818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77656bb7-0214-4b05-b552-66a8a5cf033d service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:18:58.399188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1070/2133] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:18:58 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:01.544830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481c4bc-6bb3-4a1a-8f86-de8359f9776a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:01.611041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481c4bc-6bb3-4a1a-8f86-de8359f9776a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:01.616305 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:01.616921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:01.617222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:01.617487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] PATH_INFO: `/v3/users/aa26c019cb704ddda51493c411e57a0a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:01.618624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa26c019cb704ddda51493c411e57a0a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:01.620642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:01.731189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:01.844020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa26c019cb704ddda51493c411e57a0a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:01.844674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa26c019cb704ddda51493c411e57a0a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:01.851497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa26c019cb704ddda51493c411e57a0a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:01.852126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa26c019cb704ddda51493c411e57a0a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:01.932030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa26c019cb704ddda51493c411e57a0a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:01.951365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1064/2134] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:01 2022] DELETE /identity/v3/users/aa26c019cb704ddda51493c411e57a0a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:01.956219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4bee-7dfd-414c-be16-48135ea6ceb0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:01.957002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4bee-7dfd-414c-be16-48135ea6ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:01.957279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4bee-7dfd-414c-be16-48135ea6ceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:01.977470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c3db4bee-7dfd-414c-be16-48135ea6ceb0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:01.978650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c3db4bee-7dfd-414c-be16-48135ea6ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:01.995068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:02.037790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1071/2135] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:02.047918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:02.056280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:02.112877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310d62a-5d0b-48ef-8992-a4c4892c6a64 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:02.116768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cae6d2-fb45-444b-abb2-1b9e7c736cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:02.117378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cae6d2-fb45-444b-abb2-1b9e7c736cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:02.117658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cae6d2-fb45-444b-abb2-1b9e7c736cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:02.117932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cae6d2-fb45-444b-abb2-1b9e7c736cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:02.127064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cae6d2-fb45-444b-abb2-1b9e7c736cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:02.128493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cae6d2-fb45-444b-abb2-1b9e7c736cd1 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:02.138478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1065/2136] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:02.611560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db4bee-7dfd-414c-be16-48135ea6ceb0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:02.612993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c3db4bee-7dfd-414c-be16-48135ea6ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:02.627626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db4bee-7dfd-414c-be16-48135ea6ceb0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:02.633950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:02.634478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:02.634758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:02.644731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] PATH_INFO: `/v3/projects/14d74e54e8fb44d4a97a561fd818c519` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:02.644731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d74e54e8fb44d4a97a561fd818c519)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:02.644731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:02.787415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d74e54e8fb44d4a97a561fd818c519', 'request_id': 'req-75c8e7ac-8968-4b4d-b951-4139a20ad069'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:02.895297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d74e54e8fb44d4a97a561fd818c519', 'request_id': 'req-75c8e7ac-8968-4b4d-b951-4139a20ad069'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:02.901354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d74e54e8fb44d4a97a561fd818c519', 'request_id': 'req-75c8e7ac-8968-4b4d-b951-4139a20ad069'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:02.913479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] The token cache is being invalidate because project 14d74e54e8fb44d4a97a561fd818c519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:02.913771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:02.914407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:02.915934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1072/2137] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:02 2022] DELETE /identity/v3/projects/14d74e54e8fb44d4a97a561fd818c519 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:02.919963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cae6d2-fb45-444b-abb2-1b9e7c736cd1 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:02.929769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cae6d2-fb45-444b-abb2-1b9e7c736cd1 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:02.987196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:02.987832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:02.988096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:02.988368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] PATH_INFO: `/v3/users/43388fad959a40fd9c94670fc58baf0b` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:02.989534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43388fad959a40fd9c94670fc58baf0b)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:02.991526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:03.248297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43388fad959a40fd9c94670fc58baf0b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:03.248886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43388fad959a40fd9c94670fc58baf0b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:03.250076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:03.256360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43388fad959a40fd9c94670fc58baf0b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:03.257005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43388fad959a40fd9c94670fc58baf0b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:03.372257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d9779fdc-f3b0-47d0-9fa7-c2be4f79c930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43388fad959a40fd9c94670fc58baf0b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:03.390562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1066/2138] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:02 2022] DELETE /identity/v3/users/43388fad959a40fd9c94670fc58baf0b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:03.846705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:03.848209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:03.862655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e7ac-8968-4b4d-b951-4139a20ad069 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:03.926433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:03.927098 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:03.927380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:03.927654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] PATH_INFO: `/v3/projects/d384e07649e042529d5b3391e1505b36` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:03.928772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d384e07649e042529d5b3391e1505b36)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:03.930729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:04.058599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:04.059853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd384e07649e042529d5b3391e1505b36', 'request_id': 'req-a246dfae-0389-4a0d-a008-680dda4df9d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:04.194986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd384e07649e042529d5b3391e1505b36', 'request_id': 'req-a246dfae-0389-4a0d-a008-680dda4df9d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:04.201542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd384e07649e042529d5b3391e1505b36', 'request_id': 'req-a246dfae-0389-4a0d-a008-680dda4df9d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:04.217220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] The token cache is being invalidate because project d384e07649e042529d5b3391e1505b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:04.217220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:04.217220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a246dfae-0389-4a0d-a008-680dda4df9d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:04.217220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1073/2139] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:03 2022] DELETE /identity/v3/projects/d384e07649e042529d5b3391e1505b36 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:04.221091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce473c-c103-4646-b7ff-92edaa6f5b2f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.221542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce473c-c103-4646-b7ff-92edaa6f5b2f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.221645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce473c-c103-4646-b7ff-92edaa6f5b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.241270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-06ce473c-c103-4646-b7ff-92edaa6f5b2f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:04.242461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-06ce473c-c103-4646-b7ff-92edaa6f5b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:04.303078 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1067/2140] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:04.310895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41a09a-03ac-4d26-b922-4661b8c5d690 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.311211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41a09a-03ac-4d26-b922-4661b8c5d690 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.311487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41a09a-03ac-4d26-b922-4661b8c5d690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.331477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5e41a09a-03ac-4d26-b922-4661b8c5d690 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:04.332668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5e41a09a-03ac-4d26-b922-4661b8c5d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:04.341443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1074/2141] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:04.345624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce473c-c103-4646-b7ff-92edaa6f5b2f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:04.355516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce473c-c103-4646-b7ff-92edaa6f5b2f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:04.360271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:04.360867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.361145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.361466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.362116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:04.362421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:04.364142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:04.376504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1068/2142] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:04.380429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41a09a-03ac-4d26-b922-4661b8c5d690 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:04.390373 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41a09a-03ac-4d26-b922-4661b8c5d690 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:04.395317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976662f4-bfc2-4fef-9bbc-caabec04332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:04.395912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976662f4-bfc2-4fef-9bbc-caabec04332f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.396186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976662f4-bfc2-4fef-9bbc-caabec04332f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.396456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976662f4-bfc2-4fef-9bbc-caabec04332f admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.398828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976662f4-bfc2-4fef-9bbc-caabec04332f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:04.399177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976662f4-bfc2-4fef-9bbc-caabec04332f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:04.488174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1075/2143] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:04 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:04.492256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:04.502422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f590342-a05d-4a03-b375-efeca9cd703f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:04.558956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92014fff-3b26-4f4c-8476-31af3b367868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:04.559557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92014fff-3b26-4f4c-8476-31af3b367868 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.559838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92014fff-3b26-4f4c-8476-31af3b367868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.560110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92014fff-3b26-4f4c-8476-31af3b367868 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.560780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92014fff-3b26-4f4c-8476-31af3b367868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:04.562753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92014fff-3b26-4f4c-8476-31af3b367868 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:04.676035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1069/2144] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:04 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:04.680332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976662f4-bfc2-4fef-9bbc-caabec04332f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:04.690693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976662f4-bfc2-4fef-9bbc-caabec04332f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:04.695260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8e43-7728-455b-b296-d99a243ab9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:04.695844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8e43-7728-455b-b296-d99a243ab9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.696121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8e43-7728-455b-b296-d99a243ab9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.696393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8e43-7728-455b-b296-d99a243ab9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.697037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c8e43-7728-455b-b296-d99a243ab9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:04.699310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c8e43-7728-455b-b296-d99a243ab9e2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:04.709426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1076/2145] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:04.713641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92014fff-3b26-4f4c-8476-31af3b367868 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:04.724401 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92014fff-3b26-4f4c-8476-31af3b367868 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:04.729155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31700-05ea-4be7-a95a-263ba4a42db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:04.730499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31700-05ea-4be7-a95a-263ba4a42db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.730790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31700-05ea-4be7-a95a-263ba4a42db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.731159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31700-05ea-4be7-a95a-263ba4a42db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.731807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee31700-05ea-4be7-a95a-263ba4a42db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:04.733437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee31700-05ea-4be7-a95a-263ba4a42db3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:04.744372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1070/2146] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:04.748511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8e43-7728-455b-b296-d99a243ab9e2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:04.758715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8e43-7728-455b-b296-d99a243ab9e2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:04.763401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a86dd3-dbdd-413c-a9ce-874e6e73e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:04.764050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a86dd3-dbdd-413c-a9ce-874e6e73e670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.764331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a86dd3-dbdd-413c-a9ce-874e6e73e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.764608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a86dd3-dbdd-413c-a9ce-874e6e73e670 admin admin] PATH_INFO: `/v3/projects/1f05f092537a4742b922ba5974d022bb/users/b8f8a8f44f4f47fc9cf19a1542c381a1/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.780098 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a86dd3-dbdd-413c-a9ce-874e6e73e670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f05f092537a4742b922ba5974d022bb, user_id=b8f8a8f44f4f47fc9cf19a1542c381a1, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:04.782787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a86dd3-dbdd-413c-a9ce-874e6e73e670 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:04.862585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1077/2147] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:04 2022] PUT /identity/v3/projects/1f05f092537a4742b922ba5974d022bb/users/b8f8a8f44f4f47fc9cf19a1542c381a1/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:04.872932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc79e2-7175-47c1-84ed-2218c90abc94 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:04.873301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc79e2-7175-47c1-84ed-2218c90abc94 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:04.873649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc79e2-7175-47c1-84ed-2218c90abc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:04.903284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9ddc79e2-7175-47c1-84ed-2218c90abc94 None None] MFA Rules not processed for user `b8f8a8f44f4f47fc9cf19a1542c381a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:04.904650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9ddc79e2-7175-47c1-84ed-2218c90abc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:04.924326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:04.973008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1071/2148] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:04.977602 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a86dd3-dbdd-413c-a9ce-874e6e73e670 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:05.038532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a86dd3-dbdd-413c-a9ce-874e6e73e670 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:05.043265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af14ad-b169-48c0-8281-7215042916e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:05.043859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af14ad-b169-48c0-8281-7215042916e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.044139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af14ad-b169-48c0-8281-7215042916e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.044416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af14ad-b169-48c0-8281-7215042916e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.045034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af14ad-b169-48c0-8281-7215042916e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:05.046970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af14ad-b169-48c0-8281-7215042916e7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:05.164330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1078/2149] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:04 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:05.167300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc79e2-7175-47c1-84ed-2218c90abc94 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:05.185638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc79e2-7175-47c1-84ed-2218c90abc94 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:05.244642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf56a3-5d61-4d52-8c31-e70e2bcccb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:05.245792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf56a3-5d61-4d52-8c31-e70e2bcccb3c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.246041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf56a3-5d61-4d52-8c31-e70e2bcccb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.246306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf56a3-5d61-4d52-8c31-e70e2bcccb3c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.246934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf56a3-5d61-4d52-8c31-e70e2bcccb3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:05.248939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf56a3-5d61-4d52-8c31-e70e2bcccb3c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:05.332546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1072/2150] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:05 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:05.336475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af14ad-b169-48c0-8281-7215042916e7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:05.347611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af14ad-b169-48c0-8281-7215042916e7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:05.352253 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef0751-c88f-491d-90ff-3d7b9079114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:05.356534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0751-c88f-491d-90ff-3d7b9079114e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.356534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0751-c88f-491d-90ff-3d7b9079114e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.356534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0751-c88f-491d-90ff-3d7b9079114e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.356534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef0751-c88f-491d-90ff-3d7b9079114e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:05.356534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef0751-c88f-491d-90ff-3d7b9079114e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:05.366513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1079/2151] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:05.370505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf56a3-5d61-4d52-8c31-e70e2bcccb3c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:05.380461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf56a3-5d61-4d52-8c31-e70e2bcccb3c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:05.387136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1ce5f-a0bc-4608-8978-b592909cba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:05.387136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1ce5f-a0bc-4608-8978-b592909cba87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.387136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1ce5f-a0bc-4608-8978-b592909cba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.387136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1ce5f-a0bc-4608-8978-b592909cba87 admin admin] PATH_INFO: `/v3/projects/f935f39d9ea44365a80a9744051f1547/users/ea381d2ab5094fd0af3450f357295103/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.400433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1ce5f-a0bc-4608-8978-b592909cba87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f935f39d9ea44365a80a9744051f1547, user_id=ea381d2ab5094fd0af3450f357295103, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:05.403608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1ce5f-a0bc-4608-8978-b592909cba87 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:05.475779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1073/2152] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:05 2022] PUT /identity/v3/projects/f935f39d9ea44365a80a9744051f1547/users/ea381d2ab5094fd0af3450f357295103/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:05.482065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef0751-c88f-491d-90ff-3d7b9079114e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:05.492043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef0751-c88f-491d-90ff-3d7b9079114e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:05.534510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:05.547053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212ca13-b239-4018-865d-d6502ddfd874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:05.547649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212ca13-b239-4018-865d-d6502ddfd874 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.547925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212ca13-b239-4018-865d-d6502ddfd874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.548194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212ca13-b239-4018-865d-d6502ddfd874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.548821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212ca13-b239-4018-865d-d6502ddfd874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:05.550416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212ca13-b239-4018-865d-d6502ddfd874 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:05.559791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1080/2153] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:05.564046 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1ce5f-a0bc-4608-8978-b592909cba87 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:05.575337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1ce5f-a0bc-4608-8978-b592909cba87 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:05.579961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:05.580611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.580890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.581163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] PATH_INFO: `/v3/projects/f935f39d9ea44365a80a9744051f1547/users/ea381d2ab5094fd0af3450f357295103/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.583124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f935f39d9ea44365a80a9744051f1547, user_id=ea381d2ab5094fd0af3450f357295103, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:05.585758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:05.596951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1074/2154] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:05 2022] PUT /identity/v3/projects/f935f39d9ea44365a80a9744051f1547/users/ea381d2ab5094fd0af3450f357295103/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:05.608448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765ef3b-0d8b-4109-b97f-2efdcffedad1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.608736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765ef3b-0d8b-4109-b97f-2efdcffedad1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.609015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765ef3b-0d8b-4109-b97f-2efdcffedad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.638028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7765ef3b-0d8b-4109-b97f-2efdcffedad1 None None] MFA Rules not processed for user `ea381d2ab5094fd0af3450f357295103`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:05.639249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7765ef3b-0d8b-4109-b97f-2efdcffedad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:05.706385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1081/2155] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:05.718326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:05.720102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:05.779615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389aeb-2537-4875-81f2-d75edc8b6984 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:05.783487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e261d-eab0-41c9-9f6d-538e63aee0ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:05.784095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e261d-eab0-41c9-9f6d-538e63aee0ca service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.784803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e261d-eab0-41c9-9f6d-538e63aee0ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.784803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e261d-eab0-41c9-9f6d-538e63aee0ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.793804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e261d-eab0-41c9-9f6d-538e63aee0ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:05.795278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e261d-eab0-41c9-9f6d-538e63aee0ca service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:05.804638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1075/2156] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:05.833116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765ef3b-0d8b-4109-b97f-2efdcffedad1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:05.895294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765ef3b-0d8b-4109-b97f-2efdcffedad1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:05.899874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:05.900485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:05.900763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:05.901034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] PATH_INFO: `/v3/users/b8f8a8f44f4f47fc9cf19a1542c381a1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:05.902182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8f8a8f44f4f47fc9cf19a1542c381a1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:05.904719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:06.133080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8f8a8f44f4f47fc9cf19a1542c381a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:06.140597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8f8a8f44f4f47fc9cf19a1542c381a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:06.141252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8f8a8f44f4f47fc9cf19a1542c381a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:06.141853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8f8a8f44f4f47fc9cf19a1542c381a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:06.151869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:06.155202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8f8a8f44f4f47fc9cf19a1542c381a1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:06.230996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1082/2157] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:05 2022] DELETE /identity/v3/users/b8f8a8f44f4f47fc9cf19a1542c381a1 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:06.235545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d81d-a82b-4c8d-9445-ec9710abdd6a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:06.235834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d81d-a82b-4c8d-9445-ec9710abdd6a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:06.236111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d81d-a82b-4c8d-9445-ec9710abdd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:06.255267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3e76d81d-a82b-4c8d-9445-ec9710abdd6a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:06.256471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3e76d81d-a82b-4c8d-9445-ec9710abdd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:06.266086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:06.315939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1076/2158] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:06.326975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:06.328377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:06.389095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e715ac3-1704-400f-86d5-c19f1cc7abb2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:06.392662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04fb1-3c48-46d1-9c1b-1afe0228fd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:06.393258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04fb1-3c48-46d1-9c1b-1afe0228fd39 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:06.393530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04fb1-3c48-46d1-9c1b-1afe0228fd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:06.393796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04fb1-3c48-46d1-9c1b-1afe0228fd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:06.402745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d04fb1-3c48-46d1-9c1b-1afe0228fd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:06.404182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d04fb1-3c48-46d1-9c1b-1afe0228fd39 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:06.413498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1083/2159] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:06.851577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d81d-a82b-4c8d-9445-ec9710abdd6a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:06.853039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3e76d81d-a82b-4c8d-9445-ec9710abdd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:06.870443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d81d-a82b-4c8d-9445-ec9710abdd6a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:06.870443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:06.871008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:06.871299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:06.871571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] PATH_INFO: `/v3/projects/1f05f092537a4742b922ba5974d022bb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:06.872701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f05f092537a4742b922ba5974d022bb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:06.874615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:06.982237 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f05f092537a4742b922ba5974d022bb', 'request_id': 'req-6b499127-90f1-4057-8e2f-af47924b593e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:06.987824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f05f092537a4742b922ba5974d022bb', 'request_id': 'req-6b499127-90f1-4057-8e2f-af47924b593e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:06.993982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f05f092537a4742b922ba5974d022bb', 'request_id': 'req-6b499127-90f1-4057-8e2f-af47924b593e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:07.089606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] The token cache is being invalidate because project 1f05f092537a4742b922ba5974d022bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:07.089821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:07.090466 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:07.092187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1077/2160] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:06 2022] DELETE /identity/v3/projects/1f05f092537a4742b922ba5974d022bb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:07.096201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04fb1-3c48-46d1-9c1b-1afe0228fd39 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:07.106694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04fb1-3c48-46d1-9c1b-1afe0228fd39 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:07.163650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:07.164295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:07.164573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:07.164867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] PATH_INFO: `/v3/users/ea381d2ab5094fd0af3450f357295103` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:07.166075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea381d2ab5094fd0af3450f357295103)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:07.168021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:07.393500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea381d2ab5094fd0af3450f357295103'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:07.400424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea381d2ab5094fd0af3450f357295103'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:07.401294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea381d2ab5094fd0af3450f357295103'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:07.402037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea381d2ab5094fd0af3450f357295103'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:07.414761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0eb50976-0ac2-40ef-889f-f3daf3b862aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea381d2ab5094fd0af3450f357295103'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:07.530343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1084/2161] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:07 2022] DELETE /identity/v3/users/ea381d2ab5094fd0af3450f357295103 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:08.020406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:08.021998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:08.035799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499127-90f1-4057-8e2f-af47924b593e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:08.100506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:08.101212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:08.101496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:08.101771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] PATH_INFO: `/v3/projects/f935f39d9ea44365a80a9744051f1547` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:08.103104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f935f39d9ea44365a80a9744051f1547)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:08.105270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:08.228307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:08.237154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f935f39d9ea44365a80a9744051f1547', 'request_id': 'req-46d62d57-abdc-45df-a548-2a54cc106c82'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:08.245857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f935f39d9ea44365a80a9744051f1547', 'request_id': 'req-46d62d57-abdc-45df-a548-2a54cc106c82'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:08.255846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f935f39d9ea44365a80a9744051f1547', 'request_id': 'req-46d62d57-abdc-45df-a548-2a54cc106c82'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:08.385833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] The token cache is being invalidate because project f935f39d9ea44365a80a9744051f1547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:08.386125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:08.386772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-46d62d57-abdc-45df-a548-2a54cc106c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:08.389680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1078/2162] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:08 2022] DELETE /identity/v3/projects/f935f39d9ea44365a80a9744051f1547 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:08.393600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441618-2864-4d56-93e9-46c3150b0060 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:08.394090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441618-2864-4d56-93e9-46c3150b0060 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:08.394222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441618-2864-4d56-93e9-46c3150b0060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:08.420815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2e441618-2864-4d56-93e9-46c3150b0060 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:08.422909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2e441618-2864-4d56-93e9-46c3150b0060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:08.431255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:08.509233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1085/2163] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:08.517659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3262a0-dff6-4e36-aed2-d58595dcd907 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:08.518211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3262a0-dff6-4e36-aed2-d58595dcd907 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:08.518713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3262a0-dff6-4e36-aed2-d58595dcd907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:08.541380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2e3262a0-dff6-4e36-aed2-d58595dcd907 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:08.542986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2e3262a0-dff6-4e36-aed2-d58595dcd907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:08.551502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1079/2164] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:08.556751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e441618-2864-4d56-93e9-46c3150b0060 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:08.568372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e441618-2864-4d56-93e9-46c3150b0060 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:08.573816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:08.574424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:08.577625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:08.577625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:08.577625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:08.577625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:08.578112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:08.591740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1086/2165] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:08.596192 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3262a0-dff6-4e36-aed2-d58595dcd907 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:08.615450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3262a0-dff6-4e36-aed2-d58595dcd907 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:08.616451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366cf16-ede5-4d9b-8939-98e58cdaed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:08.617254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366cf16-ede5-4d9b-8939-98e58cdaed89 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:08.617537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366cf16-ede5-4d9b-8939-98e58cdaed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:08.617810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366cf16-ede5-4d9b-8939-98e58cdaed89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:08.618505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366cf16-ede5-4d9b-8939-98e58cdaed89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:08.620569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366cf16-ede5-4d9b-8939-98e58cdaed89 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:08.790636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1080/2166] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:08 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:08.791982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:08.811807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546d6d9-f939-46c9-9cf2-3562ebc0840a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:08.891706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781e289-1060-4ef1-8a81-da8f5d7214c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:08.893014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781e289-1060-4ef1-8a81-da8f5d7214c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:08.893369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781e289-1060-4ef1-8a81-da8f5d7214c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:08.893749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781e289-1060-4ef1-8a81-da8f5d7214c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:08.894642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781e289-1060-4ef1-8a81-da8f5d7214c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:08.897605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781e289-1060-4ef1-8a81-da8f5d7214c3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:09.014019 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1087/2167] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:08 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:09.022294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366cf16-ede5-4d9b-8939-98e58cdaed89 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:09.033755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366cf16-ede5-4d9b-8939-98e58cdaed89 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:09.038456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516dee2-af12-49fa-9a85-95dab4224493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:09.039102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516dee2-af12-49fa-9a85-95dab4224493 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:09.039411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516dee2-af12-49fa-9a85-95dab4224493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:09.039647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516dee2-af12-49fa-9a85-95dab4224493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:09.040291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516dee2-af12-49fa-9a85-95dab4224493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:09.041889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516dee2-af12-49fa-9a85-95dab4224493 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:09.052135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1081/2168] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:09.056422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781e289-1060-4ef1-8a81-da8f5d7214c3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:09.066868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781e289-1060-4ef1-8a81-da8f5d7214c3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:09.071468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe796288-7b31-4784-9fb6-e09be306739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:09.072054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe796288-7b31-4784-9fb6-e09be306739d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:09.072329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe796288-7b31-4784-9fb6-e09be306739d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:09.072600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe796288-7b31-4784-9fb6-e09be306739d admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:09.073231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe796288-7b31-4784-9fb6-e09be306739d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:09.075433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe796288-7b31-4784-9fb6-e09be306739d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:09.086196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1088/2169] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:09.090729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516dee2-af12-49fa-9a85-95dab4224493 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:09.101417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516dee2-af12-49fa-9a85-95dab4224493 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:09.107536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6af29-a4b2-4cea-9685-a6a2a5a46b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:09.108187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6af29-a4b2-4cea-9685-a6a2a5a46b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:09.108465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6af29-a4b2-4cea-9685-a6a2a5a46b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:09.108740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6af29-a4b2-4cea-9685-a6a2a5a46b9b admin admin] PATH_INFO: `/v3/projects/ec47da5751fc4b4489ebe48b51dedf0d/users/6ecda0ce30dd42f888b888a200cbf6b6/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:09.125564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6af29-a4b2-4cea-9685-a6a2a5a46b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec47da5751fc4b4489ebe48b51dedf0d, user_id=6ecda0ce30dd42f888b888a200cbf6b6, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:09.131321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6af29-a4b2-4cea-9685-a6a2a5a46b9b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:09.229276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1082/2170] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:09 2022] PUT /identity/v3/projects/ec47da5751fc4b4489ebe48b51dedf0d/users/6ecda0ce30dd42f888b888a200cbf6b6/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:09.241425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0e9e9-84a4-4b87-9a93-a19daab6ae37 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:09.241748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0e9e9-84a4-4b87-9a93-a19daab6ae37 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:09.242038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0e9e9-84a4-4b87-9a93-a19daab6ae37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:09.271459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7df0e9e9-84a4-4b87-9a93-a19daab6ae37 None None] MFA Rules not processed for user `6ecda0ce30dd42f888b888a200cbf6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:09.272864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7df0e9e9-84a4-4b87-9a93-a19daab6ae37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:09.338913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1089/2171] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:09.343941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6af29-a4b2-4cea-9685-a6a2a5a46b9b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:09.428547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6af29-a4b2-4cea-9685-a6a2a5a46b9b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:09.433517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ee85b-1a7a-49fa-b50e-a69e86b81348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:09.434665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ee85b-1a7a-49fa-b50e-a69e86b81348 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:09.434962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ee85b-1a7a-49fa-b50e-a69e86b81348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:09.435233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ee85b-1a7a-49fa-b50e-a69e86b81348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:09.435853 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ee85b-1a7a-49fa-b50e-a69e86b81348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:09.437759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ee85b-1a7a-49fa-b50e-a69e86b81348 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:09.531963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1083/2172] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:09 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:09.534457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0e9e9-84a4-4b87-9a93-a19daab6ae37 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:09.547845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0e9e9-84a4-4b87-9a93-a19daab6ae37 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:09.624959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f67ea5-16f2-4ea7-833b-8e36f72a2d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:09.625570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f67ea5-16f2-4ea7-833b-8e36f72a2d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:09.625852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f67ea5-16f2-4ea7-833b-8e36f72a2d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:09.626128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f67ea5-16f2-4ea7-833b-8e36f72a2d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:09.626784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f67ea5-16f2-4ea7-833b-8e36f72a2d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:09.628858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f67ea5-16f2-4ea7-833b-8e36f72a2d9b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:09.642562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:09.751119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1090/2173] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:09 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:09.755322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ee85b-1a7a-49fa-b50e-a69e86b81348 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:09.771842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ee85b-1a7a-49fa-b50e-a69e86b81348 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:09.790543 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea379ef-48ce-428e-9ac9-373ef5d47006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:09.791312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea379ef-48ce-428e-9ac9-373ef5d47006 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:09.791666 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea379ef-48ce-428e-9ac9-373ef5d47006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:09.792014 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea379ef-48ce-428e-9ac9-373ef5d47006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:09.792754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea379ef-48ce-428e-9ac9-373ef5d47006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:09.795159 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea379ef-48ce-428e-9ac9-373ef5d47006 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:09.833408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1084/2174] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:09.844175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f67ea5-16f2-4ea7-833b-8e36f72a2d9b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:09.857938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f67ea5-16f2-4ea7-833b-8e36f72a2d9b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:09.867059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d1005-bb65-4307-bbe2-e0facaf359f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:09.867818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d1005-bb65-4307-bbe2-e0facaf359f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:09.868104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d1005-bb65-4307-bbe2-e0facaf359f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:09.868431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d1005-bb65-4307-bbe2-e0facaf359f2 admin admin] PATH_INFO: `/v3/projects/ad572d4a76aa456ab6788518e688327f/users/7cdef18e6bf841849ecec18d263083bb/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:09.901743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d1005-bb65-4307-bbe2-e0facaf359f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad572d4a76aa456ab6788518e688327f, user_id=7cdef18e6bf841849ecec18d263083bb, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:09.905344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d1005-bb65-4307-bbe2-e0facaf359f2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:10.042653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1091/2175] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:09 2022] PUT /identity/v3/projects/ad572d4a76aa456ab6788518e688327f/users/7cdef18e6bf841849ecec18d263083bb/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:10.047392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea379ef-48ce-428e-9ac9-373ef5d47006 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:10.065049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea379ef-48ce-428e-9ac9-373ef5d47006 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:10.122768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:10.142768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10bd28-29c2-431e-b8b8-568d7ae6d6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:10.143519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bd28-29c2-431e-b8b8-568d7ae6d6ca admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:10.143900 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bd28-29c2-431e-b8b8-568d7ae6d6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:10.144280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bd28-29c2-431e-b8b8-568d7ae6d6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:10.145035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10bd28-29c2-431e-b8b8-568d7ae6d6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:10.146758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10bd28-29c2-431e-b8b8-568d7ae6d6ca admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:10.157722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1085/2176] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:10.162428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d1005-bb65-4307-bbe2-e0facaf359f2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:10.173826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d1005-bb65-4307-bbe2-e0facaf359f2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:10.179372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:10.180028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:10.180479 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:10.180756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] PATH_INFO: `/v3/projects/ad572d4a76aa456ab6788518e688327f/users/7cdef18e6bf841849ecec18d263083bb/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:10.183413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad572d4a76aa456ab6788518e688327f, user_id=7cdef18e6bf841849ecec18d263083bb, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:10.186164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:10.197240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1092/2177] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:10 2022] PUT /identity/v3/projects/ad572d4a76aa456ab6788518e688327f/users/7cdef18e6bf841849ecec18d263083bb/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:10.208338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66807d0-86c8-47d2-8789-47e6f751d0f9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:10.208759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66807d0-86c8-47d2-8789-47e6f751d0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:10.209187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66807d0-86c8-47d2-8789-47e6f751d0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:10.238658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e66807d0-86c8-47d2-8789-47e6f751d0f9 None None] MFA Rules not processed for user `7cdef18e6bf841849ecec18d263083bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:10.239892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e66807d0-86c8-47d2-8789-47e6f751d0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:10.334723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1086/2178] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:10.440629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:10.443826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:10.528255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a05e87-30fa-482c-a947-347a63004fc7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:10.535730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115cdce-b88b-4719-b3ec-b3f3280adeb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:10.541359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115cdce-b88b-4719-b3ec-b3f3280adeb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:10.541359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115cdce-b88b-4719-b3ec-b3f3280adeb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:10.541359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115cdce-b88b-4719-b3ec-b3f3280adeb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:10.568392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115cdce-b88b-4719-b3ec-b3f3280adeb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:10.574965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115cdce-b88b-4719-b3ec-b3f3280adeb0 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:10.599959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1093/2179] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:11.042710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66807d0-86c8-47d2-8789-47e6f751d0f9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:11.044538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e66807d0-86c8-47d2-8789-47e6f751d0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:11.129810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66807d0-86c8-47d2-8789-47e6f751d0f9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:11.134564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:11.135212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:11.135488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:11.135757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] PATH_INFO: `/v3/users/0fc5b2b720594d65b8711ab49f396d3f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:11.137983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc5b2b720594d65b8711ab49f396d3f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:11.140410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:11.417299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc5b2b720594d65b8711ab49f396d3f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:11.417953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc5b2b720594d65b8711ab49f396d3f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:11.425173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc5b2b720594d65b8711ab49f396d3f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:11.426033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc5b2b720594d65b8711ab49f396d3f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:11.537900 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc5b2b720594d65b8711ab49f396d3f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:11.556307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1087/2180] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:11 2022] DELETE /identity/v3/users/0fc5b2b720594d65b8711ab49f396d3f => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:11.561428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2613-cda5-40a0-9bb9-9f9cab1f3b5e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:11.569269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2613-cda5-40a0-9bb9-9f9cab1f3b5e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:11.569269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d2613-cda5-40a0-9bb9-9f9cab1f3b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:11.582225 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9f0d2613-cda5-40a0-9bb9-9f9cab1f3b5e None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:11.583533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9f0d2613-cda5-40a0-9bb9-9f9cab1f3b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:11.647283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1094/2181] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:11.658436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:11.659577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:11.717610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d0531-8a66-4be9-9419-1f801d938971 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:11.721212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:11.721808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:11.722107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:11.722357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:11.732081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:11.733625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:11.743320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1088/2182] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:12.257116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:12.299251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d2613-cda5-40a0-9bb9-9f9cab1f3b5e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:12.300939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9f0d2613-cda5-40a0-9bb9-9f9cab1f3b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:12.314334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d2613-cda5-40a0-9bb9-9f9cab1f3b5e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:12.319433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:12.320066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:12.320347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:12.320620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] PATH_INFO: `/v3/projects/67a05a96d9e84c33ac2b9efbaf7935ef` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:12.321751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=67a05a96d9e84c33ac2b9efbaf7935ef)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:12.326586 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:12.380921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:12.382299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:12.395539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600681a-e961-4ad5-99e8-779c7ef7049d service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:12.400304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:12.400936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:12.401215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:12.401514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] PATH_INFO: `/v3/users/6ecda0ce30dd42f888b888a200cbf6b6` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:12.402630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ecda0ce30dd42f888b888a200cbf6b6)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:12.404783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:12.464281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67a05a96d9e84c33ac2b9efbaf7935ef', 'request_id': 'req-bb5d0085-98e7-422c-87fd-091494800c9c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.617739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67a05a96d9e84c33ac2b9efbaf7935ef', 'request_id': 'req-bb5d0085-98e7-422c-87fd-091494800c9c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.625232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67a05a96d9e84c33ac2b9efbaf7935ef', 'request_id': 'req-bb5d0085-98e7-422c-87fd-091494800c9c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.638449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] The token cache is being invalidate because project 67a05a96d9e84c33ac2b9efbaf7935ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:12.638788 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.639516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.641009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1095/2183] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:12 2022] DELETE /identity/v3/projects/67a05a96d9e84c33ac2b9efbaf7935ef => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:12.645714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:12.647163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:12.703091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ecda0ce30dd42f888b888a200cbf6b6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.703959 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ecda0ce30dd42f888b888a200cbf6b6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.711452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ecda0ce30dd42f888b888a200cbf6b6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.712115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ecda0ce30dd42f888b888a200cbf6b6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.712625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d0085-98e7-422c-87fd-091494800c9c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:12.717903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:12.718528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:12.718803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:12.719111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] PATH_INFO: `/v3/users/deadeb680ae948b999a6b37127ea2750` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:12.723302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deadeb680ae948b999a6b37127ea2750)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:12.723302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:12.826146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-bb35979c-2b46-4703-af5e-5720333a5cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ecda0ce30dd42f888b888a200cbf6b6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.845408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1089/2184] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:12 2022] DELETE /identity/v3/users/6ecda0ce30dd42f888b888a200cbf6b6 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:12.849750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba254-4ac0-48b8-bddb-2e50bf4ecbff None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:12.850117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba254-4ac0-48b8-bddb-2e50bf4ecbff None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:12.850399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba254-4ac0-48b8-bddb-2e50bf4ecbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:12.869964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2cfba254-4ac0-48b8-bddb-2e50bf4ecbff None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:12.871256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2cfba254-4ac0-48b8-bddb-2e50bf4ecbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:12.929361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1090/2185] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:12.938870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfba254-4ac0-48b8-bddb-2e50bf4ecbff None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:12.940169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2cfba254-4ac0-48b8-bddb-2e50bf4ecbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:12.969376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deadeb680ae948b999a6b37127ea2750'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.976845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deadeb680ae948b999a6b37127ea2750'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.977859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deadeb680ae948b999a6b37127ea2750'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.978726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deadeb680ae948b999a6b37127ea2750'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.992256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deadeb680ae948b999a6b37127ea2750'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:12.998270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfba254-4ac0-48b8-bddb-2e50bf4ecbff None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:13.002226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:13.002840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:13.003151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:13.003426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:13.005163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:13.017206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:13.018655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:13.029004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1091/2186] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:13.152596 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1096/2187] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:12 2022] DELETE /identity/v3/users/deadeb680ae948b999a6b37127ea2750 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:13.610418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:13.611891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:13.639434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:13.641477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22aa9f4-338f-4677-b166-149fb6ef89b3 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:13.641838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:13.741980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a4fa-1a2b-491d-b1c0-f23f257b4995 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:13.748336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:13.748971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:13.749246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:13.749515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] PATH_INFO: `/v3/projects/ea826bc8900f4d0a95c5144384c39184` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:13.750652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea826bc8900f4d0a95c5144384c39184)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:13.752797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:13.763546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:13.774464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:13.775474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:13.776671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:13.777155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] PATH_INFO: `/v3/projects/ec47da5751fc4b4489ebe48b51dedf0d` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:13.778694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec47da5751fc4b4489ebe48b51dedf0d)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:13.781825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:13.892169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea826bc8900f4d0a95c5144384c39184', 'request_id': 'req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:13.926017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec47da5751fc4b4489ebe48b51dedf0d', 'request_id': 'req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:13.931793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec47da5751fc4b4489ebe48b51dedf0d', 'request_id': 'req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:13.941136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec47da5751fc4b4489ebe48b51dedf0d', 'request_id': 'req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:13.988560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea826bc8900f4d0a95c5144384c39184', 'request_id': 'req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:13.994519 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea826bc8900f4d0a95c5144384c39184', 'request_id': 'req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.005520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] The token cache is being invalidate because project ea826bc8900f4d0a95c5144384c39184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:14.005805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.006414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2e0ad87d-b6d1-4a52-9535-3d5010b541cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.007935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1097/2188] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:13 2022] DELETE /identity/v3/projects/ea826bc8900f4d0a95c5144384c39184 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:14.013045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576e568-e525-479b-a56b-2c9507c83a70 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.013321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576e568-e525-479b-a56b-2c9507c83a70 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.013636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576e568-e525-479b-a56b-2c9507c83a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.032567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b576e568-e525-479b-a56b-2c9507c83a70 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:14.033761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b576e568-e525-479b-a56b-2c9507c83a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:14.090743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1098/2189] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:14.096737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] The token cache is being invalidate because project ec47da5751fc4b4489ebe48b51dedf0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:14.097026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.097690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.098474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b292d1a-0313-4970-af34-b5149d8b1f3a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.098761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b292d1a-0313-4970-af34-b5149d8b1f3a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.099065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b292d1a-0313-4970-af34-b5149d8b1f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.099244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1092/2190] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:13 2022] DELETE /identity/v3/projects/ec47da5751fc4b4489ebe48b51dedf0d => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:14.102835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:14.104170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:14.116235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5199c-6f63-4cc6-9d3e-87830d1ba903 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:14.117768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2b292d1a-0313-4970-af34-b5149d8b1f3a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:14.118967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2b292d1a-0313-4970-af34-b5149d8b1f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:14.121063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:14.121671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.121949 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.122256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] PATH_INFO: `/v3/users/7cdef18e6bf841849ecec18d263083bb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.123427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdef18e6bf841849ecec18d263083bb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:14.125285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:14.127044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1099/2191] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:14.130830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b292d1a-0313-4970-af34-b5149d8b1f3a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:14.140108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b292d1a-0313-4970-af34-b5149d8b1f3a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:14.144799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:14.145466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.145953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.146346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.147025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:14.147333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:14.148974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:14.161376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1100/2192] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:19:14.165693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:14.175241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62d071-27e4-4eb9-b641-c8a2e4a5e774 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:14.180020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b67c4-35b9-42b9-b4d2-135a54ab077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:14.180676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b67c4-35b9-42b9-b4d2-135a54ab077c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.180963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b67c4-35b9-42b9-b4d2-135a54ab077c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.181239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b67c4-35b9-42b9-b4d2-135a54ab077c admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.181845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b67c4-35b9-42b9-b4d2-135a54ab077c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:14.183780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b67c4-35b9-42b9-b4d2-135a54ab077c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:14.341616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1101/2193] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:14 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:14.345606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b67c4-35b9-42b9-b4d2-135a54ab077c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:14.369290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:14.382206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdef18e6bf841849ecec18d263083bb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.382787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdef18e6bf841849ecec18d263083bb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.390363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdef18e6bf841849ecec18d263083bb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.391053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdef18e6bf841849ecec18d263083bb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.412144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b67c4-35b9-42b9-b4d2-135a54ab077c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:14.417498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:14.418575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.423789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.423789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.423789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:14.423789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:14.555338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdef18e6bf841849ecec18d263083bb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:14.569114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1102/2194] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:14 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:14.572406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:14.575205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:14.587970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1093/2195] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:14 2022] DELETE /identity/v3/users/7cdef18e6bf841849ecec18d263083bb => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:14.594349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cdd1e-6fb1-410c-a866-c70a89d75dd0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:14.660647 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733b45e-8fcd-4bc8-b02b-66a1024ad9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:14.661277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733b45e-8fcd-4bc8-b02b-66a1024ad9ae admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.661506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733b45e-8fcd-4bc8-b02b-66a1024ad9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.661780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733b45e-8fcd-4bc8-b02b-66a1024ad9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.662432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733b45e-8fcd-4bc8-b02b-66a1024ad9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:14.664176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733b45e-8fcd-4bc8-b02b-66a1024ad9ae admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:14.675435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1103/2196] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:14.679418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:14.692487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303a006-caca-4854-a4c7-b872cdd0321f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:14.696705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a43f36-9c1e-4f86-a287-f974264735c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:14.697492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43f36-9c1e-4f86-a287-f974264735c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.697857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43f36-9c1e-4f86-a287-f974264735c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.698129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43f36-9c1e-4f86-a287-f974264735c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.698775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a43f36-9c1e-4f86-a287-f974264735c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:14.700406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a43f36-9c1e-4f86-a287-f974264735c1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:14.713153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1094/2197] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:14.721882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733b45e-8fcd-4bc8-b02b-66a1024ad9ae admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:14.732473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733b45e-8fcd-4bc8-b02b-66a1024ad9ae admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:14.737590 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17394bc2-b54b-463c-9fac-722fe55d94cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:14.738233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17394bc2-b54b-463c-9fac-722fe55d94cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.738510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17394bc2-b54b-463c-9fac-722fe55d94cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.738784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17394bc2-b54b-463c-9fac-722fe55d94cd admin admin] PATH_INFO: `/v3/projects/cffcf7098d204e4f87287dd2d7f1e4b0/users/9ae5dbb0148a441294d48469614e5a4c/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.755029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17394bc2-b54b-463c-9fac-722fe55d94cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffcf7098d204e4f87287dd2d7f1e4b0, user_id=9ae5dbb0148a441294d48469614e5a4c, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:14.758130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17394bc2-b54b-463c-9fac-722fe55d94cd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:14.832743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1104/2198] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:14 2022] PUT /identity/v3/projects/cffcf7098d204e4f87287dd2d7f1e4b0/users/9ae5dbb0148a441294d48469614e5a4c/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:14.843490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cf42-74bd-4513-80c2-c43f6714ed0e None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:14.843709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cf42-74bd-4513-80c2-c43f6714ed0e None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:14.844324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cf42-74bd-4513-80c2-c43f6714ed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:14.875322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-58b1cf42-74bd-4513-80c2-c43f6714ed0e None None] MFA Rules not processed for user `9ae5dbb0148a441294d48469614e5a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:14.876528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-58b1cf42-74bd-4513-80c2-c43f6714ed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:14.944916 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1095/2199] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:14.956069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17394bc2-b54b-463c-9fac-722fe55d94cd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.018806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17394bc2-b54b-463c-9fac-722fe55d94cd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.023691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d60689-1397-4c90-945d-22726267bc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.024298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60689-1397-4c90-945d-22726267bc14 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.024577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60689-1397-4c90-945d-22726267bc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.024848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60689-1397-4c90-945d-22726267bc14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.025505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d60689-1397-4c90-945d-22726267bc14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.028144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d60689-1397-4c90-945d-22726267bc14 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.058275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1cf42-74bd-4513-80c2-c43f6714ed0e None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.059760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-58b1cf42-74bd-4513-80c2-c43f6714ed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:15.086197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1cf42-74bd-4513-80c2-c43f6714ed0e None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.091947 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.092586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.092846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.093110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] PATH_INFO: `/v3/projects/ad572d4a76aa456ab6788518e688327f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.096209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad572d4a76aa456ab6788518e688327f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.096209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.161226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1105/2200] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:14 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:15.165236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d60689-1397-4c90-945d-22726267bc14 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.215526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad572d4a76aa456ab6788518e688327f', 'request_id': 'req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:15.221425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad572d4a76aa456ab6788518e688327f', 'request_id': 'req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:15.227727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad572d4a76aa456ab6788518e688327f', 'request_id': 'req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:15.228275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d60689-1397-4c90-945d-22726267bc14 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.284687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.285290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.285610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.285897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.286510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.288511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.318553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] The token cache is being invalidate because project ad572d4a76aa456ab6788518e688327f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:15.318836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:15.328779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-55f933e1-2d9f-4b4c-a517-1b48aa2b894b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:15.328779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1096/2201] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:15 2022] DELETE /identity/v3/projects/ad572d4a76aa456ab6788518e688327f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:15.328779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ab849-e1d3-479e-8831-9edf8ff4d8d8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.328779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ab849-e1d3-479e-8831-9edf8ff4d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.328779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ab849-e1d3-479e-8831-9edf8ff4d8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.347101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-665ab849-e1d3-479e-8831-9edf8ff4d8d8 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:15.348252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-665ab849-e1d3-479e-8831-9edf8ff4d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:15.356536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1097/2202] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:15.364218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c5e9-d725-4e89-91d6-0dce129bb729 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.364499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c5e9-d725-4e89-91d6-0dce129bb729 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.364832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76c5e9-d725-4e89-91d6-0dce129bb729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.383821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0f76c5e9-d725-4e89-91d6-0dce129bb729 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:15.384987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0f76c5e9-d725-4e89-91d6-0dce129bb729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:15.393943 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1098/2203] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:15.397988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76c5e9-d725-4e89-91d6-0dce129bb729 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.402894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1106/2204] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:15 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:15.403930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.405535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:15.408072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76c5e9-d725-4e89-91d6-0dce129bb729 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.413039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.413638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.413922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.414196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.415198 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:15.415331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.420622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.420729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaefa02-4663-4693-a874-2491ad1b4000 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.422093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e5c40-f5be-4f5a-9f01-3ee11376f609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.422681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e5c40-f5be-4f5a-9f01-3ee11376f609 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.422982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e5c40-f5be-4f5a-9f01-3ee11376f609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.423256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e5c40-f5be-4f5a-9f01-3ee11376f609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.423894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e5c40-f5be-4f5a-9f01-3ee11376f609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.425473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e5c40-f5be-4f5a-9f01-3ee11376f609 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.428751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1099/2205] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:15.433270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.438618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1107/2206] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:15.439579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e5c40-f5be-4f5a-9f01-3ee11376f609 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.443821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e01afd-ffce-4800-b1e2-dbe4771fcc47 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.448630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b180d4-768e-44f4-bad6-b2807c048b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.449770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b180d4-768e-44f4-bad6-b2807c048b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.449895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e5c40-f5be-4f5a-9f01-3ee11376f609 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.450047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b180d4-768e-44f4-bad6-b2807c048b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.450316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b180d4-768e-44f4-bad6-b2807c048b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.450964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b180d4-768e-44f4-bad6-b2807c048b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.453050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b180d4-768e-44f4-bad6-b2807c048b6e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.454490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186372eb-6cb6-4ce7-bdc3-439ef4dad266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.455136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186372eb-6cb6-4ce7-bdc3-439ef4dad266 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.455412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186372eb-6cb6-4ce7-bdc3-439ef4dad266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.456599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186372eb-6cb6-4ce7-bdc3-439ef4dad266 admin admin] PATH_INFO: `/v3/projects/de61a058e48848e7936ce94d85d51b38/users/43e4df9694194bfc90fff328d9203ab6/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.471409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186372eb-6cb6-4ce7-bdc3-439ef4dad266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de61a058e48848e7936ce94d85d51b38, user_id=43e4df9694194bfc90fff328d9203ab6, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.474139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186372eb-6cb6-4ce7-bdc3-439ef4dad266 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.560753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1100/2207] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:15 2022] POST /identity/v3/projects => generated 361 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:15.564794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b180d4-768e-44f4-bad6-b2807c048b6e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.607164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1108/2208] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:15 2022] PUT /identity/v3/projects/de61a058e48848e7936ce94d85d51b38/users/43e4df9694194bfc90fff328d9203ab6/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:15.612491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186372eb-6cb6-4ce7-bdc3-439ef4dad266 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.630022 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b180d4-768e-44f4-bad6-b2807c048b6e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.679182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186372eb-6cb6-4ce7-bdc3-439ef4dad266 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.685319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c470-d29a-41d8-aea9-ad93b3534a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.685926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c470-d29a-41d8-aea9-ad93b3534a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.686255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c470-d29a-41d8-aea9-ad93b3534a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.686526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c470-d29a-41d8-aea9-ad93b3534a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.687260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93c470-d29a-41d8-aea9-ad93b3534a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.688845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93c470-d29a-41d8-aea9-ad93b3534a62 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.692297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b1fb9-f1fc-4776-b129-717d8d5b2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.693023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b1fb9-f1fc-4776-b129-717d8d5b2c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.693189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b1fb9-f1fc-4776-b129-717d8d5b2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.693462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b1fb9-f1fc-4776-b129-717d8d5b2c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.694090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b1fb9-f1fc-4776-b129-717d8d5b2c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.696125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b1fb9-f1fc-4776-b129-717d8d5b2c3f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.700474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1109/2209] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:15.707671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c470-d29a-41d8-aea9-ad93b3534a62 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.721323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c470-d29a-41d8-aea9-ad93b3534a62 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.733806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3823ce4-a43b-4757-b6ed-79b20ce1a4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.734510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3823ce4-a43b-4757-b6ed-79b20ce1a4a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.734793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3823ce4-a43b-4757-b6ed-79b20ce1a4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.735345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3823ce4-a43b-4757-b6ed-79b20ce1a4a6 admin admin] PATH_INFO: `/v3/projects/de61a058e48848e7936ce94d85d51b38/users/43e4df9694194bfc90fff328d9203ab6/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.737736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3823ce4-a43b-4757-b6ed-79b20ce1a4a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de61a058e48848e7936ce94d85d51b38, user_id=43e4df9694194bfc90fff328d9203ab6, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.740622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3823ce4-a43b-4757-b6ed-79b20ce1a4a6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.763736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1110/2210] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:15 2022] PUT /identity/v3/projects/de61a058e48848e7936ce94d85d51b38/users/43e4df9694194bfc90fff328d9203ab6/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:15.766705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47226e9d-7c83-40ac-a7e3-3cd8a3d32b9d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.767105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47226e9d-7c83-40ac-a7e3-3cd8a3d32b9d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.767447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47226e9d-7c83-40ac-a7e3-3cd8a3d32b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.782297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1101/2211] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:15 2022] POST /identity/v3/users => generated 281 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:15.782297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b1fb9-f1fc-4776-b129-717d8d5b2c3f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.794828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b1fb9-f1fc-4776-b129-717d8d5b2c3f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.804818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-47226e9d-7c83-40ac-a7e3-3cd8a3d32b9d None None] MFA Rules not processed for user `43e4df9694194bfc90fff328d9203ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:15.806397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-47226e9d-7c83-40ac-a7e3-3cd8a3d32b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:15.880735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c3a74-40be-4474-ba12-f56291fd74b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.881353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3a74-40be-4474-ba12-f56291fd74b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.881630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3a74-40be-4474-ba12-f56291fd74b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.881897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3a74-40be-4474-ba12-f56291fd74b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.882545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c3a74-40be-4474-ba12-f56291fd74b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.884212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c3a74-40be-4474-ba12-f56291fd74b5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.895231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1102/2212] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:15.899074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c3a74-40be-4474-ba12-f56291fd74b5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.900504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1111/2213] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:15.909098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c3a74-40be-4474-ba12-f56291fd74b5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.911810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47226e9d-7c83-40ac-a7e3-3cd8a3d32b9d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.913136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-47226e9d-7c83-40ac-a7e3-3cd8a3d32b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:15.914039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa557def-948f-4947-8ad4-e41492065989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.914634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa557def-948f-4947-8ad4-e41492065989 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.914915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa557def-948f-4947-8ad4-e41492065989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.915221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa557def-948f-4947-8ad4-e41492065989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.915860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa557def-948f-4947-8ad4-e41492065989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.917506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa557def-948f-4947-8ad4-e41492065989 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.928055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1103/2214] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:15.932178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa557def-948f-4947-8ad4-e41492065989 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:15.942598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa557def-948f-4947-8ad4-e41492065989 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.947377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9a641-0aa1-430b-9127-a3f42cb18b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.948006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9a641-0aa1-430b-9127-a3f42cb18b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.948287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9a641-0aa1-430b-9127-a3f42cb18b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.948570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9a641-0aa1-430b-9127-a3f42cb18b03 admin admin] PATH_INFO: `/v3/projects/76d5a09298d345a9a0abf776fe167217/users/12dbf3915919431cb616c6b756dc7344/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.964388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9a641-0aa1-430b-9127-a3f42cb18b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d5a09298d345a9a0abf776fe167217, user_id=12dbf3915919431cb616c6b756dc7344, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.967117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9a641-0aa1-430b-9127-a3f42cb18b03 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.971086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47226e9d-7c83-40ac-a7e3-3cd8a3d32b9d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:15.974856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c49dde-6aaf-4c33-a068-1415853f7c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:15.975556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49dde-6aaf-4c33-a068-1415853f7c5b service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:15.975899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49dde-6aaf-4c33-a068-1415853f7c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:15.976254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49dde-6aaf-4c33-a068-1415853f7c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:15.985202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c49dde-6aaf-4c33-a068-1415853f7c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:15.986661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c49dde-6aaf-4c33-a068-1415853f7c5b service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:15.987772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:15.996680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1112/2215] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:16.043635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1104/2216] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:15 2022] PUT /identity/v3/projects/76d5a09298d345a9a0abf776fe167217/users/12dbf3915919431cb616c6b756dc7344/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:16.055198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3887692-ea24-4285-a4dd-5cb0c5c158f8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:16.055497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3887692-ea24-4285-a4dd-5cb0c5c158f8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:16.055792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3887692-ea24-4285-a4dd-5cb0c5c158f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:16.075255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9a641-0aa1-430b-9127-a3f42cb18b03 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:16.122780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e3887692-ea24-4285-a4dd-5cb0c5c158f8 None None] MFA Rules not processed for user `12dbf3915919431cb616c6b756dc7344`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:16.124737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e3887692-ea24-4285-a4dd-5cb0c5c158f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:16.148821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9a641-0aa1-430b-9127-a3f42cb18b03 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:16.153596 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:16.154226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:16.154506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:16.154778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] PATH_INFO: `/v3/users/9ae5dbb0148a441294d48469614e5a4c` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:16.155989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae5dbb0148a441294d48469614e5a4c)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:16.158005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:16.207364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1113/2217] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:16.247103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3887692-ea24-4285-a4dd-5cb0c5c158f8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:16.306902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3887692-ea24-4285-a4dd-5cb0c5c158f8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:16.312300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636a2c-0f74-43bf-b9e7-0fe05e6dbb3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:16.318649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636a2c-0f74-43bf-b9e7-0fe05e6dbb3a service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:16.318649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636a2c-0f74-43bf-b9e7-0fe05e6dbb3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:16.318649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636a2c-0f74-43bf-b9e7-0fe05e6dbb3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:16.323049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5636a2c-0f74-43bf-b9e7-0fe05e6dbb3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:16.324574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5636a2c-0f74-43bf-b9e7-0fe05e6dbb3a service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:16.333578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1114/2218] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:16.348800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636a2c-0f74-43bf-b9e7-0fe05e6dbb3a service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:16.360118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636a2c-0f74-43bf-b9e7-0fe05e6dbb3a service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:16.364972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:16.365572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:16.365848 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:16.366119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] PATH_INFO: `/v3/users/12dbf3915919431cb616c6b756dc7344` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:16.367294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=12dbf3915919431cb616c6b756dc7344)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:16.369201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:16.469936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae5dbb0148a441294d48469614e5a4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.470589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae5dbb0148a441294d48469614e5a4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.477695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae5dbb0148a441294d48469614e5a4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.478368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae5dbb0148a441294d48469614e5a4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.480486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:16.569671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12dbf3915919431cb616c6b756dc7344'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.576616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12dbf3915919431cb616c6b756dc7344'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.577329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12dbf3915919431cb616c6b756dc7344'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.578018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12dbf3915919431cb616c6b756dc7344'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.585385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-be8e869a-adab-45b3-9cf2-5b90c8591217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae5dbb0148a441294d48469614e5a4c'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.591101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86f00ae2-bb10-483b-85a0-38bd361b01ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12dbf3915919431cb616c6b756dc7344'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:16.603656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1105/2219] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:16 2022] DELETE /identity/v3/users/9ae5dbb0148a441294d48469614e5a4c => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:16.608512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d090fe-45d6-484c-855c-ea7176f9631d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:16.608790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d090fe-45d6-484c-855c-ea7176f9631d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:16.609059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d090fe-45d6-484c-855c-ea7176f9631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:16.627018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b3d090fe-45d6-484c-855c-ea7176f9631d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:16.628197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b3d090fe-45d6-484c-855c-ea7176f9631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:16.690870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1106/2220] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:16.700602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d090fe-45d6-484c-855c-ea7176f9631d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:16.702083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b3d090fe-45d6-484c-855c-ea7176f9631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:16.711075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1115/2221] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:16 2022] DELETE /identity/v3/users/12dbf3915919431cb616c6b756dc7344 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:16.716131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268b9ff-b0a3-475e-a58a-b545c51fbd4b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:16.716414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268b9ff-b0a3-475e-a58a-b545c51fbd4b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:16.716773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268b9ff-b0a3-475e-a58a-b545c51fbd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:16.736641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9268b9ff-b0a3-475e-a58a-b545c51fbd4b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:16.737842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9268b9ff-b0a3-475e-a58a-b545c51fbd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:16.774225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d090fe-45d6-484c-855c-ea7176f9631d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:16.814903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1116/2222] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:16.830506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268b9ff-b0a3-475e-a58a-b545c51fbd4b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:16.832191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:16.832906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:16.833189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:16.833461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:16.840676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268b9ff-b0a3-475e-a58a-b545c51fbd4b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:16.844201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:16.844338 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:16.844828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:16.845105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:16.845374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:16.845660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:16.854918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:16.855709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1107/2223] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:16.856379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:16.867046 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1117/2224] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:17.576165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:17.576165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:17.577202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:17.578675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:17.585699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ffe53-4c17-4b27-b9c2-a89e2f60990c service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:17.590758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:17.599866 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19282-bcde-44a7-8f69-c38838fb8cac service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:17.605267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:17.606065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:17.606361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:17.606686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] PATH_INFO: `/v3/projects/76d5a09298d345a9a0abf776fe167217` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:17.606798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:17.606997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:17.607279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] PATH_INFO: `/v3/projects/cffcf7098d204e4f87287dd2d7f1e4b0` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:17.607906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d5a09298d345a9a0abf776fe167217)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:17.608544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cffcf7098d204e4f87287dd2d7f1e4b0)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:17.609890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:17.610616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:17.685203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:17.736092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d5a09298d345a9a0abf776fe167217', 'request_id': 'req-81684cb1-b22b-4e1f-a32b-cc255553411a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.737638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cffcf7098d204e4f87287dd2d7f1e4b0', 'request_id': 'req-0304056e-ce4e-43d5-844b-3c88589c3b12'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.744872 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cffcf7098d204e4f87287dd2d7f1e4b0', 'request_id': 'req-0304056e-ce4e-43d5-844b-3c88589c3b12'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.754774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cffcf7098d204e4f87287dd2d7f1e4b0', 'request_id': 'req-0304056e-ce4e-43d5-844b-3c88589c3b12'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.798915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d5a09298d345a9a0abf776fe167217', 'request_id': 'req-81684cb1-b22b-4e1f-a32b-cc255553411a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.805410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d5a09298d345a9a0abf776fe167217', 'request_id': 'req-81684cb1-b22b-4e1f-a32b-cc255553411a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.817752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] The token cache is being invalidate because project 76d5a09298d345a9a0abf776fe167217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:17.817989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.818696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-81684cb1-b22b-4e1f-a32b-cc255553411a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.820394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1118/2225] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:17 2022] DELETE /identity/v3/projects/76d5a09298d345a9a0abf776fe167217 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:17.824935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021b4b-e1f9-4d1a-9791-68bee5a60a2d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:17.825221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021b4b-e1f9-4d1a-9791-68bee5a60a2d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:17.825498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021b4b-e1f9-4d1a-9791-68bee5a60a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:17.847027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-74021b4b-e1f9-4d1a-9791-68bee5a60a2d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:17.848329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-74021b4b-e1f9-4d1a-9791-68bee5a60a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:17.907629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1119/2226] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:17.915276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87b7bb-0dd6-4999-a27d-28dabc92279f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:17.915561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87b7bb-0dd6-4999-a27d-28dabc92279f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:17.915834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87b7bb-0dd6-4999-a27d-28dabc92279f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:17.919098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] The token cache is being invalidate because project cffcf7098d204e4f87287dd2d7f1e4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:17.920875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.920875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:17.921436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1108/2227] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:17 2022] DELETE /identity/v3/projects/cffcf7098d204e4f87287dd2d7f1e4b0 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:17.925251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:17.926548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:17.936061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bf87b7bb-0dd6-4999-a27d-28dabc92279f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:17.937238 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bf87b7bb-0dd6-4999-a27d-28dabc92279f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:17.945772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1120/2228] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:17.961434 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87b7bb-0dd6-4999-a27d-28dabc92279f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:17.971992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87b7bb-0dd6-4999-a27d-28dabc92279f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:17.983580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:17.985905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:17.987459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:17.995614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:17.995614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:17.995614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:17.995614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:17.995614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:18.006308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1121/2229] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:18.010723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:18.025456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88734c1-f956-4e63-b36d-f8220fb92976 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:18.032932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590e7ee-93fd-4947-866c-ed3542432f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:18.035754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590e7ee-93fd-4947-866c-ed3542432f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.035754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590e7ee-93fd-4947-866c-ed3542432f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.035754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590e7ee-93fd-4947-866c-ed3542432f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.035754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590e7ee-93fd-4947-866c-ed3542432f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:18.037844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304056e-ce4e-43d5-844b-3c88589c3b12 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:18.038244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590e7ee-93fd-4947-866c-ed3542432f08 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:18.043325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:18.043986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.044279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.044555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] PATH_INFO: `/v3/users/43e4df9694194bfc90fff328d9203ab6` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.045825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e4df9694194bfc90fff328d9203ab6)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:18.047854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:18.206572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1122/2230] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:18 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:18.210933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590e7ee-93fd-4947-866c-ed3542432f08 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:18.276818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590e7ee-93fd-4947-866c-ed3542432f08 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:18.281699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:18.282409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.282640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.282912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.283691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:18.285520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:18.323517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e4df9694194bfc90fff328d9203ab6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:18.324171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e4df9694194bfc90fff328d9203ab6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:18.330888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e4df9694194bfc90fff328d9203ab6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:18.332719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e4df9694194bfc90fff328d9203ab6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:18.438774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e4df9694194bfc90fff328d9203ab6'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:18.444970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1123/2231] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:18 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:18.449547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:18.451532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:18.458671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1109/2232] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:17 2022] DELETE /identity/v3/users/43e4df9694194bfc90fff328d9203ab6 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:18.464503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4cf3f-e9b9-4e9b-81cf-4349f0970eb9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:18.528508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f68c6c-4ea8-489d-87a4-e3e5ac907b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:18.529092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f68c6c-4ea8-489d-87a4-e3e5ac907b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.531654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f68c6c-4ea8-489d-87a4-e3e5ac907b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.531654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f68c6c-4ea8-489d-87a4-e3e5ac907b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.531654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f68c6c-4ea8-489d-87a4-e3e5ac907b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:18.532176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f68c6c-4ea8-489d-87a4-e3e5ac907b7e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:18.545768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1124/2233] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:18.547266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:18.561593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc87f09-01a0-4ea3-b669-e181eedad8ce admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:18.567409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a860-6561-4413-8af3-fc045443a48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:18.568027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a860-6561-4413-8af3-fc045443a48e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.568302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a860-6561-4413-8af3-fc045443a48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.568571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a860-6561-4413-8af3-fc045443a48e admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.569245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a860-6561-4413-8af3-fc045443a48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:18.570855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a860-6561-4413-8af3-fc045443a48e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:18.582320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1110/2234] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:18.586686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f68c6c-4ea8-489d-87a4-e3e5ac907b7e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:18.597475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f68c6c-4ea8-489d-87a4-e3e5ac907b7e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:18.602183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:18.602883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.603178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.603415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] PATH_INFO: `/v3/projects/57bec21bdcc0435f9cfd1c935bc66cab/users/c64637d2aa4b43a68b89908074eb1375/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.619298 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bec21bdcc0435f9cfd1c935bc66cab, user_id=c64637d2aa4b43a68b89908074eb1375, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:18.622099 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:18.716780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1125/2235] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:18 2022] PUT /identity/v3/projects/57bec21bdcc0435f9cfd1c935bc66cab/users/c64637d2aa4b43a68b89908074eb1375/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:18.727355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae893f-514d-47f3-94db-f2f3e26c6692 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.727771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae893f-514d-47f3-94db-f2f3e26c6692 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.728227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae893f-514d-47f3-94db-f2f3e26c6692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.759986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0fae893f-514d-47f3-94db-f2f3e26c6692 None None] MFA Rules not processed for user `c64637d2aa4b43a68b89908074eb1375`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:18.761806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0fae893f-514d-47f3-94db-f2f3e26c6692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:18.832065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:18.833043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:18.833900 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1111/2236] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:18.838634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae893f-514d-47f3-94db-f2f3e26c6692 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:18.902070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6d110-b669-48f4-b527-efec3fe775f2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:18.904410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae893f-514d-47f3-94db-f2f3e26c6692 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:18.906684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:18.907323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.907606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.907879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] PATH_INFO: `/v3/users/7a0b9b0492634d1a90819d5086d59378` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.909050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0b9b0492634d1a90819d5086d59378)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:18.909369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:18.909971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:18.910292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:18.910580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:18.911289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:18.911408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:18.913300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:19.011648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1112/2237] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:18 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:19.012501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:19.014050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.079977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5e0d1-10fc-4a99-ae91-73a80d7992cc admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:19.117384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:19.117384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.117384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.117384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] PATH_INFO: `/v3/projects/de61a058e48848e7936ce94d85d51b38` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.117760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de61a058e48848e7936ce94d85d51b38)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:19.117760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:19.171945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0b9b0492634d1a90819d5086d59378'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.178619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0b9b0492634d1a90819d5086d59378'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.179293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0b9b0492634d1a90819d5086d59378'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.179905 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0b9b0492634d1a90819d5086d59378'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.192169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0b9b0492634d1a90819d5086d59378'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.233153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de61a058e48848e7936ce94d85d51b38', 'request_id': 'req-af59423a-a004-461a-854b-c11c120eede0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.239033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de61a058e48848e7936ce94d85d51b38', 'request_id': 'req-af59423a-a004-461a-854b-c11c120eede0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.246587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de61a058e48848e7936ce94d85d51b38', 'request_id': 'req-af59423a-a004-461a-854b-c11c120eede0'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.312902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1126/2238] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:18 2022] DELETE /identity/v3/users/7a0b9b0492634d1a90819d5086d59378 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:19.313876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:19.315330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.374288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:19.383952 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b59fe-0054-4700-8715-905bb8a64aba admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:19.389123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f4d48-ad6d-4e1d-930e-84381e875ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:19.389808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f4d48-ad6d-4e1d-930e-84381e875ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.390163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f4d48-ad6d-4e1d-930e-84381e875ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.390539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f4d48-ad6d-4e1d-930e-84381e875ccb admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.391562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f4d48-ad6d-4e1d-930e-84381e875ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:19.395681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f4d48-ad6d-4e1d-930e-84381e875ccb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:19.405576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] The token cache is being invalidate because project de61a058e48848e7936ce94d85d51b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:19.405974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.407057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.413709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1113/2239] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:19 2022] DELETE /identity/v3/projects/de61a058e48848e7936ce94d85d51b38 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:19.424431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:19.424431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.475650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:19.492235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59423a-a004-461a-854b-c11c120eede0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:19.497010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:19.498669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.498669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.498669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] PATH_INFO: `/v3/users/650ab868bb9c4d63b4fff14936ceee0f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.499473 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=650ab868bb9c4d63b4fff14936ceee0f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:19.501406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:19.524682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1127/2240] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:19 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:19.526412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af637e36-b762-414b-acf8-96cbd24ef62c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.526791 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af637e36-b762-414b-acf8-96cbd24ef62c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.527151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af637e36-b762-414b-acf8-96cbd24ef62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.546653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-af637e36-b762-414b-acf8-96cbd24ef62c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:19.547939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-af637e36-b762-414b-acf8-96cbd24ef62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.555792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1128/2241] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:19.556930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a1b89-4584-49c0-8cd5-5828d370e718 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.557213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a1b89-4584-49c0-8cd5-5828d370e718 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.557486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a1b89-4584-49c0-8cd5-5828d370e718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.580151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3a8a1b89-4584-49c0-8cd5-5828d370e718 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:19.581347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3a8a1b89-4584-49c0-8cd5-5828d370e718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.593023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1129/2242] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:19.594153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a1b89-4584-49c0-8cd5-5828d370e718 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:19.595553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3a8a1b89-4584-49c0-8cd5-5828d370e718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.608701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a1b89-4584-49c0-8cd5-5828d370e718 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:19.613761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:19.614367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.614650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.614921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.615629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:19.618299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:19.628816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1130/2243] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:19.632559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:19.634279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.699110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cc9eb-a777-4f08-9b14-3a05979daa66 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:19.703950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935dc932-3a3f-406f-94e3-18bca9aa21fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:19.704612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dc932-3a3f-406f-94e3-18bca9aa21fd service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.704896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dc932-3a3f-406f-94e3-18bca9aa21fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.705293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dc932-3a3f-406f-94e3-18bca9aa21fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.717690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935dc932-3a3f-406f-94e3-18bca9aa21fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:19.719211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935dc932-3a3f-406f-94e3-18bca9aa21fd service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:19.731904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1131/2244] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:19.733043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286ca4-1240-45b7-8db2-1b721463e1fe None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.733331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286ca4-1240-45b7-8db2-1b721463e1fe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.733610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286ca4-1240-45b7-8db2-1b721463e1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.758364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-59286ca4-1240-45b7-8db2-1b721463e1fe None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:19.760025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-59286ca4-1240-45b7-8db2-1b721463e1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.774329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1132/2245] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:19.779164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286ca4-1240-45b7-8db2-1b721463e1fe None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:19.783338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '650ab868bb9c4d63b4fff14936ceee0f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.784654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '650ab868bb9c4d63b4fff14936ceee0f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.795586 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286ca4-1240-45b7-8db2-1b721463e1fe None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:19.806968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '650ab868bb9c4d63b4fff14936ceee0f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.806968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '650ab868bb9c4d63b4fff14936ceee0f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.814887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:19.815080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.815310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.815585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.816245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:19.817981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:19.856329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1133/2246] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:19.861429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:19.861429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.893014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b8b2d-4d0e-4d0d-9dfd-7cb3014f2573 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:19.903668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c252054-2cd8-4bd2-a572-3f8c8332e4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:19.904708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252054-2cd8-4bd2-a572-3f8c8332e4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.905159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252054-2cd8-4bd2-a572-3f8c8332e4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.905427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c252054-2cd8-4bd2-a572-3f8c8332e4fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.906115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c252054-2cd8-4bd2-a572-3f8c8332e4fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:19.906425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c252054-2cd8-4bd2-a572-3f8c8332e4fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:19.908620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c252054-2cd8-4bd2-a572-3f8c8332e4fd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:19.909780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '650ab868bb9c4d63b4fff14936ceee0f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:19.934030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1114/2247] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:19 2022] DELETE /identity/v3/users/650ab868bb9c4d63b4fff14936ceee0f => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:19.935176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:19.936781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:19.942706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1134/2248] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:19.944284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2d3a9-e839-4152-b779-86bab57a0389 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:19.944610 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2d3a9-e839-4152-b779-86bab57a0389 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:19.946035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2d3a9-e839-4152-b779-86bab57a0389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:19.988757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4ca2d3a9-e839-4152-b779-86bab57a0389 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:19.989991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4ca2d3a9-e839-4152-b779-86bab57a0389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:20.044140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ceeea6-0744-466e-86eb-f042d948cb84 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.049160 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.050888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.051410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.051878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] PATH_INFO: `/v3/projects/247e95e31f5c4ad9ab436710394c1f5f/users/2ee78389a9d04f3a8396d3b8d05637c9/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.074876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=247e95e31f5c4ad9ab436710394c1f5f, user_id=2ee78389a9d04f3a8396d3b8d05637c9, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.077987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:20.164650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1135/2249] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:20.166480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2d3a9-e839-4152-b779-86bab57a0389 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:20.179035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1115/2250] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:19 2022] PUT /identity/v3/projects/247e95e31f5c4ad9ab436710394c1f5f/users/2ee78389a9d04f3a8396d3b8d05637c9/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:20.184550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:20.186886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:20.188419 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:20.194743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2d3a9-e839-4152-b779-86bab57a0389 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.275280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6d76-ffa2-4860-b1bb-029fecd471f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.275280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6d76-ffa2-4860-b1bb-029fecd471f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.275280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6d76-ffa2-4860-b1bb-029fecd471f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.275998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b6d76-ffa2-4860-b1bb-029fecd471f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.277043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b6d76-ffa2-4860-b1bb-029fecd471f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.280102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b6d76-ffa2-4860-b1bb-029fecd471f9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:20.307396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20932181-afc7-41ba-b653-7fec1487bd5f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.312322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5566f93-e9a0-4b7b-83e3-2dbfa2cc7be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.313206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5566f93-e9a0-4b7b-83e3-2dbfa2cc7be6 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.313707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5566f93-e9a0-4b7b-83e3-2dbfa2cc7be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.314220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5566f93-e9a0-4b7b-83e3-2dbfa2cc7be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.331360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5566f93-e9a0-4b7b-83e3-2dbfa2cc7be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.332862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5566f93-e9a0-4b7b-83e3-2dbfa2cc7be6 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:20.343641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1116/2251] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:20.344924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf8041-c43b-415b-9a54-774f858f3618 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.345212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf8041-c43b-415b-9a54-774f858f3618 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.345519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf8041-c43b-415b-9a54-774f858f3618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.382153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d9cf8041-c43b-415b-9a54-774f858f3618 None None] MFA Rules not processed for user `2ee78389a9d04f3a8396d3b8d05637c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:20.383418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d9cf8041-c43b-415b-9a54-774f858f3618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:20.395172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1136/2252] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:20 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:20.402677 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6d76-ffa2-4860-b1bb-029fecd471f9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:20.492587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1117/2253] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:20.493776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf8041-c43b-415b-9a54-774f858f3618 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:20.495779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d9cf8041-c43b-415b-9a54-774f858f3618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:20.517399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf8041-c43b-415b-9a54-774f858f3618 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.520809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b6d76-ffa2-4860-b1bb-029fecd471f9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.523727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.524665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.525042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.525381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] PATH_INFO: `/v3/projects/b56c6fc64aef466baaa3981132efe579` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.526465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68a305-7731-45d1-b178-b81d3b3041c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.526595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b56c6fc64aef466baaa3981132efe579)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.527461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68a305-7731-45d1-b178-b81d3b3041c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.527844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68a305-7731-45d1-b178-b81d3b3041c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.528216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68a305-7731-45d1-b178-b81d3b3041c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.528571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:20.529627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68a305-7731-45d1-b178-b81d3b3041c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.533256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68a305-7731-45d1-b178-b81d3b3041c1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:20.680489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1137/2254] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:20 2022] POST /identity/v3/users => generated 294 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:20.682341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68a305-7731-45d1-b178-b81d3b3041c1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:20.696321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68a305-7731-45d1-b178-b81d3b3041c1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.707845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b56c6fc64aef466baaa3981132efe579', 'request_id': 'req-99a79803-279c-4d58-a8b7-67e4efa74158'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:20.716245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b56c6fc64aef466baaa3981132efe579', 'request_id': 'req-99a79803-279c-4d58-a8b7-67e4efa74158'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:20.742502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b56c6fc64aef466baaa3981132efe579', 'request_id': 'req-99a79803-279c-4d58-a8b7-67e4efa74158'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:20.770193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:20.785549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da207cc5-7c8e-4c49-bb11-af8d626aa6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.786221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da207cc5-7c8e-4c49-bb11-af8d626aa6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.786507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da207cc5-7c8e-4c49-bb11-af8d626aa6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.786781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da207cc5-7c8e-4c49-bb11-af8d626aa6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.787564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da207cc5-7c8e-4c49-bb11-af8d626aa6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.789489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da207cc5-7c8e-4c49-bb11-af8d626aa6d4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:20.798519 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] The token cache is being invalidate because project b56c6fc64aef466baaa3981132efe579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:20.799415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:20.800083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:20.805002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1118/2255] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:20 2022] DELETE /identity/v3/projects/b56c6fc64aef466baaa3981132efe579 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:20.806179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:20.808295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:20.820874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a79803-279c-4d58-a8b7-67e4efa74158 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.825555 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.826141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.826418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.826683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.827356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.828994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:20.841516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1119/2256] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:20.842305 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:20.844133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:20.856778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef82157-598b-40d8-88aa-2ddbee666a0c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.861387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.861962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.862256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.862509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] PATH_INFO: `/v3/users/80d977b6686542f4adf87b0006cfbf8a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.863622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d977b6686542f4adf87b0006cfbf8a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.865490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:20.923619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1138/2257] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:20 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:20.927155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da207cc5-7c8e-4c49-bb11-af8d626aa6d4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:20.990471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da207cc5-7c8e-4c49-bb11-af8d626aa6d4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:20.995785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:20.996623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:20.996901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:20.997167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:20.997802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:20.999432 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:21.012962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1139/2258] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:21.012962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:21.014342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:21.027370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da78b7d-83be-4636-9899-554d0f763b81 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:21.031868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:21.032459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.032733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.033004 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] PATH_INFO: `/v3/projects/5183d9ba6cb340b4a5c7ce03162c6e64` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.034151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5183d9ba6cb340b4a5c7ce03162c6e64)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:21.036430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:21.039011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:21.114326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d977b6686542f4adf87b0006cfbf8a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.115006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d977b6686542f4adf87b0006cfbf8a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.121869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d977b6686542f4adf87b0006cfbf8a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.122495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d977b6686542f4adf87b0006cfbf8a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.145783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5183d9ba6cb340b4a5c7ce03162c6e64', 'request_id': 'req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.180052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d977b6686542f4adf87b0006cfbf8a'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.198045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1120/2259] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:20 2022] DELETE /identity/v3/users/80d977b6686542f4adf87b0006cfbf8a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:21.201307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:21.202171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:21.249145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5183d9ba6cb340b4a5c7ce03162c6e64', 'request_id': 'req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.257468 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5183d9ba6cb340b4a5c7ce03162c6e64', 'request_id': 'req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.271524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] The token cache is being invalidate because project 5183d9ba6cb340b4a5c7ce03162c6e64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:21.271721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.272526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.277960 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1140/2260] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:21 2022] DELETE /identity/v3/projects/5183d9ba6cb340b4a5c7ce03162c6e64 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:21.281283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:21.281671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6020d9-369d-4923-9786-1fd232bed5f8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:21.282894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:21.286401 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:21.287031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.287421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.287602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.288181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:21.290327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:21.357121 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8f055-49c3-4faa-9179-bc7b83acfe6e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:21.362405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:21.363236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.363458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.363724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] PATH_INFO: `/v3/projects/b0dcfeefe62548739498a5d05777b7fa/users/f513167636f14c378427837903d3eb05/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.381728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1121/2261] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:21 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:21.381728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:21.381728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:21.383059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0dcfeefe62548739498a5d05777b7fa, user_id=f513167636f14c378427837903d3eb05, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:21.386435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:21.394970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8348602-f6bd-42f4-ac8b-c1fb60a535e3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:21.400025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:21.401182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.405307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.405307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] PATH_INFO: `/v3/users/e3d077a6615f48a690ca12d806401930` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.405307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d077a6615f48a690ca12d806401930)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:21.405307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:21.459948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1141/2262] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:21 2022] PUT /identity/v3/projects/b0dcfeefe62548739498a5d05777b7fa/users/f513167636f14c378427837903d3eb05/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:21.461031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:21.470503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:21.536477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78323e78-6620-40a6-bf8b-17243ad98c13 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:21.541101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62af3a-ca01-4be3-adaf-292d649e57d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:21.541704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62af3a-ca01-4be3-adaf-292d649e57d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.542093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62af3a-ca01-4be3-adaf-292d649e57d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.542251 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62af3a-ca01-4be3-adaf-292d649e57d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.542882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62af3a-ca01-4be3-adaf-292d649e57d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:21.544500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62af3a-ca01-4be3-adaf-292d649e57d2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:21.554391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1142/2263] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:21.555632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c99cc7-62ae-45fe-93a2-97146544e60a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.555914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c99cc7-62ae-45fe-93a2-97146544e60a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.556186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c99cc7-62ae-45fe-93a2-97146544e60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.569658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:21.586597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a9c99cc7-62ae-45fe-93a2-97146544e60a None None] MFA Rules not processed for user `f513167636f14c378427837903d3eb05`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:21.588556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a9c99cc7-62ae-45fe-93a2-97146544e60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:21.652543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1143/2264] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:21.653376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c99cc7-62ae-45fe-93a2-97146544e60a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:21.663634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c99cc7-62ae-45fe-93a2-97146544e60a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:21.668804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:21.670674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.670674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.670674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] PATH_INFO: `/v3/projects/fa026f44e9714c978737af132a7674c3/users/474f138355df4f11a2f7faee97b7ae4f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.674223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d077a6615f48a690ca12d806401930'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.674880 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d077a6615f48a690ca12d806401930'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.682485 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d077a6615f48a690ca12d806401930'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.683185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d077a6615f48a690ca12d806401930'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.686074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa026f44e9714c978737af132a7674c3, user_id=474f138355df4f11a2f7faee97b7ae4f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:21.690182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:21.768887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d077a6615f48a690ca12d806401930'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:21.788424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1122/2265] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:21 2022] DELETE /identity/v3/users/e3d077a6615f48a690ca12d806401930 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:21.789248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:21.790742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:21.811712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1144/2266] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:21 2022] PUT /identity/v3/projects/fa026f44e9714c978737af132a7674c3/users/474f138355df4f11a2f7faee97b7ae4f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:21.812692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:21.818262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:21.860491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73415a5a-1537-4d26-911e-f882f0ae01b0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:21.914420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152e066-2492-4894-993f-e9e7224f9239 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:21.920341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:21.921193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.921527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.921962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] PATH_INFO: `/v3/projects/ab00b264001746b0b535282adcff618c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.922871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:21.923138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab00b264001746b0b535282adcff618c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:21.923514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:21.923797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:21.924069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:21.924694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:21.925079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:21.926591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.092897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1123/2267] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:21 2022] POST /identity/v3/projects => generated 387 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:22.094005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.095668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.099680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab00b264001746b0b535282adcff618c', 'request_id': 'req-2316f045-6f3b-46e1-813b-d47d7a4e197a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.162330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b6c3-6c9d-4109-bc1e-29ee4ccddc81 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.169021 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b54f8-a428-48f6-aa99-d7e3e33a7493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.169632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b54f8-a428-48f6-aa99-d7e3e33a7493 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.174435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b54f8-a428-48f6-aa99-d7e3e33a7493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.174435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b54f8-a428-48f6-aa99-d7e3e33a7493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.174435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b54f8-a428-48f6-aa99-d7e3e33a7493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.174435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b54f8-a428-48f6-aa99-d7e3e33a7493 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.183684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1124/2268] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:22.184433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b54f8-a428-48f6-aa99-d7e3e33a7493 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.194446 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b54f8-a428-48f6-aa99-d7e3e33a7493 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.198016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab00b264001746b0b535282adcff618c', 'request_id': 'req-2316f045-6f3b-46e1-813b-d47d7a4e197a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.204728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab00b264001746b0b535282adcff618c', 'request_id': 'req-2316f045-6f3b-46e1-813b-d47d7a4e197a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.208054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072fd36-49af-4360-9eab-04d5c6a3f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.208054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072fd36-49af-4360-9eab-04d5c6a3f33f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.208054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072fd36-49af-4360-9eab-04d5c6a3f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.208054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072fd36-49af-4360-9eab-04d5c6a3f33f admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.208440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072fd36-49af-4360-9eab-04d5c6a3f33f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.209618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072fd36-49af-4360-9eab-04d5c6a3f33f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.219141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] The token cache is being invalidate because project ab00b264001746b0b535282adcff618c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:22.219539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.220322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.225148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1145/2269] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:21 2022] DELETE /identity/v3/projects/ab00b264001746b0b535282adcff618c => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:22.225148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.226588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.295477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1125/2270] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:22 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:22.298495 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316f045-6f3b-46e1-813b-d47d7a4e197a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.300084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c57c7-c1db-4d4c-a3ef-a16cb82bbab6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.300437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c57c7-c1db-4d4c-a3ef-a16cb82bbab6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.303076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c57c7-c1db-4d4c-a3ef-a16cb82bbab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.304307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.305160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.305611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.306087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] PATH_INFO: `/v3/projects/fa026f44e9714c978737af132a7674c3/users/474f138355df4f11a2f7faee97b7ae4f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.308825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa026f44e9714c978737af132a7674c3, user_id=474f138355df4f11a2f7faee97b7ae4f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.311898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.326274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1146/2271] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:22 2022] PUT /identity/v3/projects/fa026f44e9714c978737af132a7674c3/users/474f138355df4f11a2f7faee97b7ae4f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:22.327322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.329302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.337638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-006c57c7-c1db-4d4c-a3ef-a16cb82bbab6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:22.339064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-006c57c7-c1db-4d4c-a3ef-a16cb82bbab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.350148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:22.407039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c996cb2-33c9-4df8-9205-f05075163c0a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.409778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1126/2272] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:22.411156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c83c3-5c4d-45d4-bb84-810ad892ccf0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.411437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c83c3-5c4d-45d4-bb84-810ad892ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.411707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c83c3-5c4d-45d4-bb84-810ad892ccf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.416830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916eae-6e3c-43c7-b750-28d997612c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.417445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916eae-6e3c-43c7-b750-28d997612c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.417726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916eae-6e3c-43c7-b750-28d997612c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.418024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916eae-6e3c-43c7-b750-28d997612c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.418686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916eae-6e3c-43c7-b750-28d997612c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.420443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916eae-6e3c-43c7-b750-28d997612c8a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.438079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1147/2273] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:22.438079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bce55-c1b9-471c-b3f7-43a26058840c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.438079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bce55-c1b9-471c-b3f7-43a26058840c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.438079 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bce55-c1b9-471c-b3f7-43a26058840c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.440769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a35c83c3-5c4d-45d4-bb84-810ad892ccf0 None None] MFA Rules not processed for user `474f138355df4f11a2f7faee97b7ae4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:22.441967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a35c83c3-5c4d-45d4-bb84-810ad892ccf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.458216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-170bce55-c1b9-471c-b3f7-43a26058840c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:22.459793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-170bce55-c1b9-471c-b3f7-43a26058840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.470719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1148/2274] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:22.471581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bce55-c1b9-471c-b3f7-43a26058840c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.483542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bce55-c1b9-471c-b3f7-43a26058840c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.488986 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.489669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.489945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.490242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] PATH_INFO: `/v3/projects/d4340692b9c34cf29c67e493804964c7/users/c55dca5dcbbd458ca6e7289b0233cdab/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.510141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4340692b9c34cf29c67e493804964c7, user_id=c55dca5dcbbd458ca6e7289b0233cdab, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.512966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.529270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1127/2275] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:22.531803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c83c3-5c4d-45d4-bb84-810ad892ccf0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.543234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c83c3-5c4d-45d4-bb84-810ad892ccf0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.547975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.548585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.548864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.549136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.549879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:22.550176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.551810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.564516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1128/2276] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:22.565548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.567535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.626527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1149/2277] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:22 2022] PUT /identity/v3/projects/d4340692b9c34cf29c67e493804964c7/users/c55dca5dcbbd458ca6e7289b0233cdab/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:22.629456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.630397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab620-8dad-4e95-888a-063b4b90559c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.632113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.684781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.685745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.686188 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.686462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.688452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.704316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a85d2-6275-4c47-a8a7-5e26de99e968 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.709628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.710291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.710567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.710840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] PATH_INFO: `/v3/projects/8352a01036e04d20b9ad1dcdecae41ea` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.712006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8352a01036e04d20b9ad1dcdecae41ea)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.714575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.752567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.754001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.763174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1129/2278] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:22.763974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.775449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2110b1-40d1-4228-ab07-028b433fecec service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.781604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5ad56-8975-4c53-9cdf-216f330383b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.782335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5ad56-8975-4c53-9cdf-216f330383b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.782612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5ad56-8975-4c53-9cdf-216f330383b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.782883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5ad56-8975-4c53-9cdf-216f330383b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.783701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5ad56-8975-4c53-9cdf-216f330383b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.786614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5ad56-8975-4c53-9cdf-216f330383b1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:22.844545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8352a01036e04d20b9ad1dcdecae41ea', 'request_id': 'req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.857706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:22.927276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1130/2279] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:22 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:22.928121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5ad56-8975-4c53-9cdf-216f330383b1 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.945753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8352a01036e04d20b9ad1dcdecae41ea', 'request_id': 'req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.952427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8352a01036e04d20b9ad1dcdecae41ea', 'request_id': 'req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.964414 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] The token cache is being invalidate because project 8352a01036e04d20b9ad1dcdecae41ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:22.964709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.965372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:22.966976 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1150/2280] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:22 2022] DELETE /identity/v3/projects/8352a01036e04d20b9ad1dcdecae41ea => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:22.967750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:22.969081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:22.991360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5ad56-8975-4c53-9cdf-216f330383b1 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:22.996004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:22.996628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:22.996904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:22.997172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:22.997809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:22.999436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.009613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1131/2281] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:23.010462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.014088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.030277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958edbe0-2747-4564-99e9-35806a2de60d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.033599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b6114-4d05-4fbe-9aae-f7e2b4f290b9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.036748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c7b1e-c359-4904-8d98-f707c791b637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.037363 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c7b1e-c359-4904-8d98-f707c791b637 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.037705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c7b1e-c359-4904-8d98-f707c791b637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.038058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c7b1e-c359-4904-8d98-f707c791b637 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.038685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c7b1e-c359-4904-8d98-f707c791b637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.039309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.039938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.040246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.040520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] PATH_INFO: `/v3/users/42c6e15c008847ff9c2481df240f050a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.040711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c7b1e-c359-4904-8d98-f707c791b637 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.041839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c6e15c008847ff9c2481df240f050a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.043811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.158986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1132/2282] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:23 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:23.159689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba7683-5e1e-44ca-a8f6-264c223c9e81 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.159977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba7683-5e1e-44ca-a8f6-264c223c9e81 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.160247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba7683-5e1e-44ca-a8f6-264c223c9e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.179746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ebba7683-5e1e-44ca-a8f6-264c223c9e81 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:23.181458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ebba7683-5e1e-44ca-a8f6-264c223c9e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.198579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1133/2283] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:23.198579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba7683-5e1e-44ca-a8f6-264c223c9e81 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.198579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ebba7683-5e1e-44ca-a8f6-264c223c9e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.204063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba7683-5e1e-44ca-a8f6-264c223c9e81 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.208965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a787e-684b-4312-9042-6b421992f726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.209605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a787e-684b-4312-9042-6b421992f726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.209876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a787e-684b-4312-9042-6b421992f726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.210142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a787e-684b-4312-9042-6b421992f726 admin admin] PATH_INFO: `/v3/projects/d4340692b9c34cf29c67e493804964c7/users/c55dca5dcbbd458ca6e7289b0233cdab/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.212064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a787e-684b-4312-9042-6b421992f726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4340692b9c34cf29c67e493804964c7, user_id=c55dca5dcbbd458ca6e7289b0233cdab, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.214742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a787e-684b-4312-9042-6b421992f726 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.225113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1134/2284] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:23 2022] PUT /identity/v3/projects/d4340692b9c34cf29c67e493804964c7/users/c55dca5dcbbd458ca6e7289b0233cdab/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:23.232632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a787e-684b-4312-9042-6b421992f726 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.256903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c6e15c008847ff9c2481df240f050a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:23.264455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c6e15c008847ff9c2481df240f050a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:23.265132 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c6e15c008847ff9c2481df240f050a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:23.265790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c6e15c008847ff9c2481df240f050a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:23.277870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c6e15c008847ff9c2481df240f050a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:23.296635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a787e-684b-4312-9042-6b421992f726 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.301297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32c187-bf25-4aef-b181-40c1cf809e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.301892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c187-bf25-4aef-b181-40c1cf809e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.302167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c187-bf25-4aef-b181-40c1cf809e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.302437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32c187-bf25-4aef-b181-40c1cf809e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.303102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32c187-bf25-4aef-b181-40c1cf809e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.304735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32c187-bf25-4aef-b181-40c1cf809e6d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.318276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1135/2285] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:23.318276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0488e-72e8-4f61-8f4d-44ab0112ed77 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.318276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0488e-72e8-4f61-8f4d-44ab0112ed77 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.318276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0488e-72e8-4f61-8f4d-44ab0112ed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.336119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ddb0488e-72e8-4f61-8f4d-44ab0112ed77 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:23.337334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ddb0488e-72e8-4f61-8f4d-44ab0112ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.345339 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1136/2286] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:23.346462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeb023-668c-4f84-8ac3-a58b01567739 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.346742 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeb023-668c-4f84-8ac3-a58b01567739 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.347044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeb023-668c-4f84-8ac3-a58b01567739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.374216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-48eeb023-668c-4f84-8ac3-a58b01567739 None None] MFA Rules not processed for user `c55dca5dcbbd458ca6e7289b0233cdab`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:23.375462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-48eeb023-668c-4f84-8ac3-a58b01567739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.396737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1151/2287] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:22 2022] DELETE /identity/v3/users/42c6e15c008847ff9c2481df240f050a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:23.399945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.399945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.445454 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1137/2288] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:23.447258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eeb023-668c-4f84-8ac3-a58b01567739 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.459023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eeb023-668c-4f84-8ac3-a58b01567739 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.467459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d67214-3505-40cb-8355-1dd91da8b93a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.473384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.474165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.474512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.474855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.476006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:23.476328 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aac262-1b41-479c-96b9-a6d2da780469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.476465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.477062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aac262-1b41-479c-96b9-a6d2da780469 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.477412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aac262-1b41-479c-96b9-a6d2da780469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.477778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aac262-1b41-479c-96b9-a6d2da780469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.478423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.478612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aac262-1b41-479c-96b9-a6d2da780469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.480241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aac262-1b41-479c-96b9-a6d2da780469 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.498906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1152/2289] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:23.498906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f1213-e011-4871-a212-091507a5bea3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.499428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1138/2290] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:23.499535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f1213-e011-4871-a212-091507a5bea3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.499535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f1213-e011-4871-a212-091507a5bea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.499729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.505161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.517221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-367f1213-e011-4871-a212-091507a5bea3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:23.518782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-367f1213-e011-4871-a212-091507a5bea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.534824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1153/2291] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:23.535655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f1213-e011-4871-a212-091507a5bea3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.545580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f1213-e011-4871-a212-091507a5bea3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.551017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.551666 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.551941 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.552211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] PATH_INFO: `/v3/projects/c04cc118470a457cb2f9d891f64a6d0d/users/a53baa179e3a4a7eb06f088783d014fb/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.571537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdbb71-4b6d-41ad-98ef-f89418876ce7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.575234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04cc118470a457cb2f9d891f64a6d0d, user_id=a53baa179e3a4a7eb06f088783d014fb, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.575234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.575430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c513a64-35df-4cc8-943b-0f8bad742bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.575977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c513a64-35df-4cc8-943b-0f8bad742bcc service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.576308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c513a64-35df-4cc8-943b-0f8bad742bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.576583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c513a64-35df-4cc8-943b-0f8bad742bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.640270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c513a64-35df-4cc8-943b-0f8bad742bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.641722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c513a64-35df-4cc8-943b-0f8bad742bcc service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.651690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1139/2292] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:23.652495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c513a64-35df-4cc8-943b-0f8bad742bcc service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.662858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c513a64-35df-4cc8-943b-0f8bad742bcc service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.681384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1154/2293] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:23 2022] PUT /identity/v3/projects/c04cc118470a457cb2f9d891f64a6d0d/users/a53baa179e3a4a7eb06f088783d014fb/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:23.681384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.682752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2747da-f0f6-447b-bc66-9330a4a3d60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.683461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2747da-f0f6-447b-bc66-9330a4a3d60e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.683797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2747da-f0f6-447b-bc66-9330a4a3d60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.684226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2747da-f0f6-447b-bc66-9330a4a3d60e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.684925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2747da-f0f6-447b-bc66-9330a4a3d60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.686156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.689193 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2747da-f0f6-447b-bc66-9330a4a3d60e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.756424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780be0d-c1d5-4aa8-b280-48cdc400bbfb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.764663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea51403-8f7f-4f1e-8d42-90a9c0b68c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.764663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea51403-8f7f-4f1e-8d42-90a9c0b68c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.764663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea51403-8f7f-4f1e-8d42-90a9c0b68c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.764663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea51403-8f7f-4f1e-8d42-90a9c0b68c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.819302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1140/2294] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:23 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:23.820638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f082f9-3fb5-414e-9599-a54658699df1 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.820990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f082f9-3fb5-414e-9599-a54658699df1 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.821374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f082f9-3fb5-414e-9599-a54658699df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.844676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea51403-8f7f-4f1e-8d42-90a9c0b68c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.846136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea51403-8f7f-4f1e-8d42-90a9c0b68c2a service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.851328 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-54f082f9-3fb5-414e-9599-a54658699df1 None None] MFA Rules not processed for user `a53baa179e3a4a7eb06f088783d014fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:23.853087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-54f082f9-3fb5-414e-9599-a54658699df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.856646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1155/2295] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:23.857542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea51403-8f7f-4f1e-8d42-90a9c0b68c2a service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.869279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea51403-8f7f-4f1e-8d42-90a9c0b68c2a service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.928695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1141/2296] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:23.928695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f082f9-3fb5-414e-9599-a54658699df1 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:23.928695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-54f082f9-3fb5-414e-9599-a54658699df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:23.936880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3366589-42a2-4aab-8f2f-f9f331af68d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.936880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3366589-42a2-4aab-8f2f-f9f331af68d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:23.936880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3366589-42a2-4aab-8f2f-f9f331af68d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:23.936880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3366589-42a2-4aab-8f2f-f9f331af68d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:23.937306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3366589-42a2-4aab-8f2f-f9f331af68d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:23.939157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3366589-42a2-4aab-8f2f-f9f331af68d1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:23.957046 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:23.994341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f082f9-3fb5-414e-9599-a54658699df1 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:23.999281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:23.999965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.000241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.000512 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] PATH_INFO: `/v3/users/f513167636f14c378427837903d3eb05` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.001721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f513167636f14c378427837903d3eb05)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.003888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:24.020470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1156/2297] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:23 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:24.021412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3366589-42a2-4aab-8f2f-f9f331af68d1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:24.078095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3366589-42a2-4aab-8f2f-f9f331af68d1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:24.082067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e89c23-0505-49d4-b49a-d50271e98706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:24.082578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e89c23-0505-49d4-b49a-d50271e98706 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.082858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e89c23-0505-49d4-b49a-d50271e98706 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.083431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e89c23-0505-49d4-b49a-d50271e98706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.093681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e89c23-0505-49d4-b49a-d50271e98706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.095192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e89c23-0505-49d4-b49a-d50271e98706 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:24.110528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1157/2298] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:24.110528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e89c23-0505-49d4-b49a-d50271e98706 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:24.120437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e89c23-0505-49d4-b49a-d50271e98706 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:24.125118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d4362-a583-4668-9394-e0680722d498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:24.125709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d4362-a583-4668-9394-e0680722d498 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.126010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d4362-a583-4668-9394-e0680722d498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.126443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d4362-a583-4668-9394-e0680722d498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.127299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d4362-a583-4668-9394-e0680722d498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.129064 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d4362-a583-4668-9394-e0680722d498 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:24.139178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1158/2299] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:24.143139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d4362-a583-4668-9394-e0680722d498 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:24.153571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d4362-a583-4668-9394-e0680722d498 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:24.158242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8616bc2-6480-4fa6-b10c-08e319690efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:24.158827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616bc2-6480-4fa6-b10c-08e319690efa admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.159134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616bc2-6480-4fa6-b10c-08e319690efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.159404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616bc2-6480-4fa6-b10c-08e319690efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.160036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8616bc2-6480-4fa6-b10c-08e319690efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.161621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8616bc2-6480-4fa6-b10c-08e319690efa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:24.174659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1159/2300] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:24.178800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8616bc2-6480-4fa6-b10c-08e319690efa admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:24.189620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8616bc2-6480-4fa6-b10c-08e319690efa admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:24.195449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:24.196282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.196678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.197052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] PATH_INFO: `/v3/projects/ea123710d8664ef88bcb0a4b005f3c22/users/d56bcd7b11684160b8ab80eb9ea72195/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.221108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f513167636f14c378427837903d3eb05'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.222037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f513167636f14c378427837903d3eb05'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.225582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea123710d8664ef88bcb0a4b005f3c22, user_id=d56bcd7b11684160b8ab80eb9ea72195, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.228958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:24.237922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f513167636f14c378427837903d3eb05'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.237922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f513167636f14c378427837903d3eb05'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.325484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-3e0dc93b-804e-4bf0-99ba-f20fa7d56750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f513167636f14c378427837903d3eb05'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.338588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1160/2301] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:24 2022] PUT /identity/v3/projects/ea123710d8664ef88bcb0a4b005f3c22/users/d56bcd7b11684160b8ab80eb9ea72195/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:24.341853 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:24.344404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:24.362266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1142/2302] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:23 2022] DELETE /identity/v3/users/f513167636f14c378427837903d3eb05 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:24.364954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bba94-f1d8-4293-9a0d-a800af9174af None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.364954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bba94-f1d8-4293-9a0d-a800af9174af None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.364954 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bba94-f1d8-4293-9a0d-a800af9174af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.419055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8e0bba94-f1d8-4293-9a0d-a800af9174af None None] MFA Rules not processed for user `d56bcd7b11684160b8ab80eb9ea72195`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:24.419055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8e0bba94-f1d8-4293-9a0d-a800af9174af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:24.436574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21514-3ef3-4f76-b30a-ad50d40ad24a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:24.487057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1143/2303] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:24.488559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6242985-a8e4-4d1a-b192-8936023f6269 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.489496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6242985-a8e4-4d1a-b192-8936023f6269 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.489828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6242985-a8e4-4d1a-b192-8936023f6269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.496937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:24.497789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.498669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.498962 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] PATH_INFO: `/v3/projects/086e5c78010f4bbc944a03bf9b86a17e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.500283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] RBAC: Authorizing `identity:delete_project(project_id=086e5c78010f4bbc944a03bf9b86a17e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.503345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:24.510187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c6242985-a8e4-4d1a-b192-8936023f6269 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:24.511464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c6242985-a8e4-4d1a-b192-8936023f6269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:24.519398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1144/2304] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:24.520184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6242985-a8e4-4d1a-b192-8936023f6269 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:24.521548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c6242985-a8e4-4d1a-b192-8936023f6269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:24.574284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:24.584164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6242985-a8e4-4d1a-b192-8936023f6269 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:24.588815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:24.589420 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.589697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.589969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.598319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.599791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:24.608664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1145/2305] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:24.609460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:24.610726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:24.645192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '086e5c78010f4bbc944a03bf9b86a17e', 'request_id': 'req-b18fbced-c49a-4b63-b022-8b8979eb672d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.677641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4f770-c9a2-4d3c-9b4e-3ce3177c7e4f service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:24.710785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '086e5c78010f4bbc944a03bf9b86a17e', 'request_id': 'req-b18fbced-c49a-4b63-b022-8b8979eb672d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.716729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '086e5c78010f4bbc944a03bf9b86a17e', 'request_id': 'req-b18fbced-c49a-4b63-b022-8b8979eb672d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.728240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:24.728843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] The token cache is being invalidate because project 086e5c78010f4bbc944a03bf9b86a17e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:24.729140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.729270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.729523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.729819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:24.729935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.731737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1161/2306] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:24 2022] DELETE /identity/v3/projects/086e5c78010f4bbc944a03bf9b86a17e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:24.731858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:24.735609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:24.738269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:24.799447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.800875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:24.811086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1146/2307] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:24.811620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fbced-c49a-4b63-b022-8b8979eb672d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:24.817536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:24.818203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:24.818484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:24.818763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] PATH_INFO: `/v3/users/a1adef3b2f1742c0b985acf98a5e2a02` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:24.820053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1adef3b2f1742c0b985acf98a5e2a02)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:24.822498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:25.050023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1adef3b2f1742c0b985acf98a5e2a02'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.061781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1adef3b2f1742c0b985acf98a5e2a02'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.063822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1adef3b2f1742c0b985acf98a5e2a02'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.064460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1adef3b2f1742c0b985acf98a5e2a02'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.081045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1adef3b2f1742c0b985acf98a5e2a02'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.209310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1162/2308] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:24 2022] DELETE /identity/v3/users/a1adef3b2f1742c0b985acf98a5e2a02 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:25.317416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:25.319236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:25.333929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7365-d3dd-460e-8ead-3062cee0a8f1 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:25.341778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:25.352758 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:25.354269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:25.411235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:25.412217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:25.412599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:25.413066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] PATH_INFO: `/v3/projects/b0dcfeefe62548739498a5d05777b7fa` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:25.414364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0dcfeefe62548739498a5d05777b7fa)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:25.416592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:25.428209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00695374-6d6d-4ee6-b554-debc6227eaa4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:25.435774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:25.437057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:25.437351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:25.437626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:25.439560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:25.507401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:25.509444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:25.520444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1163/2309] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:25.570387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0dcfeefe62548739498a5d05777b7fa', 'request_id': 'req-7e477ff2-8250-467b-bffb-35be386fe244'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.586088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0dcfeefe62548739498a5d05777b7fa', 'request_id': 'req-7e477ff2-8250-467b-bffb-35be386fe244'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.594259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0dcfeefe62548739498a5d05777b7fa', 'request_id': 'req-7e477ff2-8250-467b-bffb-35be386fe244'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.693905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] The token cache is being invalidate because project b0dcfeefe62548739498a5d05777b7fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:25.694307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.695387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.697751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1147/2310] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:25 2022] DELETE /identity/v3/projects/b0dcfeefe62548739498a5d05777b7fa => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:25.701622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:25.703176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:25.718928 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489e842-3043-4d43-b433-955853f03b1c service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:25.777405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:25.777963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:25.778242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:25.778514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] PATH_INFO: `/v3/users/c55dca5dcbbd458ca6e7289b0233cdab` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:25.779665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c55dca5dcbbd458ca6e7289b0233cdab)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:25.781648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:25.828675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:25.829657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:25.845145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e477ff2-8250-467b-bffb-35be386fe244 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:25.854502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:25.855178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:25.855469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:25.855904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] PATH_INFO: `/v3/projects/7517a59b487844c79257dc07fe546367` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:25.857442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7517a59b487844c79257dc07fe546367)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:25.859695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:25.992129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c55dca5dcbbd458ca6e7289b0233cdab'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:25.996595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7517a59b487844c79257dc07fe546367', 'request_id': 'req-7adf4b85-b626-4245-9ce4-4218d3887912'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.000751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c55dca5dcbbd458ca6e7289b0233cdab'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.001462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c55dca5dcbbd458ca6e7289b0233cdab'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.002076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c55dca5dcbbd458ca6e7289b0233cdab'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.002346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7517a59b487844c79257dc07fe546367', 'request_id': 'req-7adf4b85-b626-4245-9ce4-4218d3887912'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.009140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7517a59b487844c79257dc07fe546367', 'request_id': 'req-7adf4b85-b626-4245-9ce4-4218d3887912'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.015346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c55dca5dcbbd458ca6e7289b0233cdab'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.126273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] The token cache is being invalidate because project 7517a59b487844c79257dc07fe546367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:26.127213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.127921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-7adf4b85-b626-4245-9ce4-4218d3887912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:26.133366 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1148/2311] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:25 2022] DELETE /identity/v3/projects/7517a59b487844c79257dc07fe546367 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:26.134701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc91aa3-c7ba-470b-bffc-b3121c42efd0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.135048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc91aa3-c7ba-470b-bffc-b3121c42efd0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.135264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc91aa3-c7ba-470b-bffc-b3121c42efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.152135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:26.157686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bdc91aa3-c7ba-470b-bffc-b3121c42efd0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:26.159031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bdc91aa3-c7ba-470b-bffc-b3121c42efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:26.219228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1149/2312] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:26.221904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1164/2313] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:25 2022] DELETE /identity/v3/users/c55dca5dcbbd458ca6e7289b0233cdab => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:26.227591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e462a4b-6f46-48e2-8544-8e6197f6c67b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.227871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e462a4b-6f46-48e2-8544-8e6197f6c67b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.228129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e462a4b-6f46-48e2-8544-8e6197f6c67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.247520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4e462a4b-6f46-48e2-8544-8e6197f6c67b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:26.248757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4e462a4b-6f46-48e2-8544-8e6197f6c67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:26.310407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1150/2314] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:26.315097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:26.316529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:26.329510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a1cd-2d40-474d-877b-d699c38d0d89 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:26.334732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:26.335418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.335694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.336607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.337278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:26.337563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:26.339183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:26.350832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1165/2315] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:26.360929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e462a4b-6f46-48e2-8544-8e6197f6c67b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:26.370447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e462a4b-6f46-48e2-8544-8e6197f6c67b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:26.375142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:26.375738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.376019 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.376288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.376905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:26.378809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:26.516220 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1151/2316] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:26 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:26.517093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:26.530833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecd6b-a9ab-4ba3-a390-855d47a7b48c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:26.602775 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e59574-9205-4fd2-b637-45bcf2bdabfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:26.603410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e59574-9205-4fd2-b637-45bcf2bdabfc admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.603690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e59574-9205-4fd2-b637-45bcf2bdabfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.603968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e59574-9205-4fd2-b637-45bcf2bdabfc admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.604878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e59574-9205-4fd2-b637-45bcf2bdabfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:26.606505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e59574-9205-4fd2-b637-45bcf2bdabfc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:26.732972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:26.735214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:26.751520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038549f-31b7-4402-8f9e-e63b9acc0b5e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:26.756894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:26.769239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.769239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.769239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] PATH_INFO: `/v3/projects/d4340692b9c34cf29c67e493804964c7` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.769239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4340692b9c34cf29c67e493804964c7)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:26.769239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:26.770088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1166/2317] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:26 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:26.772343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e59574-9205-4fd2-b637-45bcf2bdabfc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:26.783312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e59574-9205-4fd2-b637-45bcf2bdabfc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:26.788945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd846b15-3bef-4907-94bd-84a91f283c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:26.790114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd846b15-3bef-4907-94bd-84a91f283c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.790403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd846b15-3bef-4907-94bd-84a91f283c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.790676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd846b15-3bef-4907-94bd-84a91f283c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.791358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd846b15-3bef-4907-94bd-84a91f283c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:26.793525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd846b15-3bef-4907-94bd-84a91f283c61 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:26.806979 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1167/2318] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:26.809369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd846b15-3bef-4907-94bd-84a91f283c61 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:26.820428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd846b15-3bef-4907-94bd-84a91f283c61 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:26.825159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a5167-8897-41a1-a8ce-2ac2362e0d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:26.825744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a5167-8897-41a1-a8ce-2ac2362e0d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.826025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a5167-8897-41a1-a8ce-2ac2362e0d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.826295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a5167-8897-41a1-a8ce-2ac2362e0d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.826923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a5167-8897-41a1-a8ce-2ac2362e0d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:26.828609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a5167-8897-41a1-a8ce-2ac2362e0d68 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:26.838921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1168/2319] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:26.852835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a5167-8897-41a1-a8ce-2ac2362e0d68 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:26.862286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a5167-8897-41a1-a8ce-2ac2362e0d68 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:26.867003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7403a8-b890-415a-96ab-1f9adfe55fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:26.868179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7403a8-b890-415a-96ab-1f9adfe55fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:26.868462 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7403a8-b890-415a-96ab-1f9adfe55fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:26.868777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7403a8-b890-415a-96ab-1f9adfe55fb9 admin admin] PATH_INFO: `/v3/projects/f5d0ee0087dd4032b371164ebef2973b/users/0ac3c8a923a24a99ad680cec81d8eb23/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:26.884690 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7403a8-b890-415a-96ab-1f9adfe55fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d0ee0087dd4032b371164ebef2973b, user_id=0ac3c8a923a24a99ad680cec81d8eb23, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:26.887567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7403a8-b890-415a-96ab-1f9adfe55fb9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:27.047684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1169/2320] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:26 2022] PUT /identity/v3/projects/f5d0ee0087dd4032b371164ebef2973b/users/0ac3c8a923a24a99ad680cec81d8eb23/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:27.055318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4340692b9c34cf29c67e493804964c7', 'request_id': 'req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:27.059139 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221011fc-d0c1-47e4-b005-ca98b606d111 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.059421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221011fc-d0c1-47e4-b005-ca98b606d111 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.059715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221011fc-d0c1-47e4-b005-ca98b606d111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.061329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4340692b9c34cf29c67e493804964c7', 'request_id': 'req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:27.067948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4340692b9c34cf29c67e493804964c7', 'request_id': 'req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:27.089626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-221011fc-d0c1-47e4-b005-ca98b606d111 None None] MFA Rules not processed for user `0ac3c8a923a24a99ad680cec81d8eb23`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:27.089626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-221011fc-d0c1-47e4-b005-ca98b606d111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:27.144736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] The token cache is being invalidate because project d4340692b9c34cf29c67e493804964c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:27.145105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:27.146003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ac912f30-3e1f-4c6a-b6b2-45b71a0d28c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:27.151988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1152/2321] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:26 2022] DELETE /identity/v3/projects/d4340692b9c34cf29c67e493804964c7 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:27.153479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28ae4-c0f4-421a-8698-3d5a985eb7b7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.153936 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28ae4-c0f4-421a-8698-3d5a985eb7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.154368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28ae4-c0f4-421a-8698-3d5a985eb7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.163194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1170/2322] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:27.181669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ceb28ae4-c0f4-421a-8698-3d5a985eb7b7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:27.182905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ceb28ae4-c0f4-421a-8698-3d5a985eb7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:27.248357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1153/2323] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:27.256416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215413ff-3554-4ce7-bf94-f4d52d172bf0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.256904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215413ff-3554-4ce7-bf94-f4d52d172bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.256984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215413ff-3554-4ce7-bf94-f4d52d172bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.276663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb28ae4-c0f4-421a-8698-3d5a985eb7b7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:27.278314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ceb28ae4-c0f4-421a-8698-3d5a985eb7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:27.297326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-215413ff-3554-4ce7-bf94-f4d52d172bf0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:27.298607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-215413ff-3554-4ce7-bf94-f4d52d172bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:27.304282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:27.317882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1171/2324] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:27.319241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215413ff-3554-4ce7-bf94-f4d52d172bf0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:27.340094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215413ff-3554-4ce7-bf94-f4d52d172bf0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:27.349246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:27.350423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.350909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.352680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.354299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:27.355052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:27.361870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb28ae4-c0f4-421a-8698-3d5a985eb7b7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:27.366027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd91b3-14b4-4e7e-abbb-b05d61d7c416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:27.366714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd91b3-14b4-4e7e-abbb-b05d61d7c416 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.367613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd91b3-14b4-4e7e-abbb-b05d61d7c416 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.367964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd91b3-14b4-4e7e-abbb-b05d61d7c416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.376923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:27.378717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd91b3-14b4-4e7e-abbb-b05d61d7c416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:27.380513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd91b3-14b4-4e7e-abbb-b05d61d7c416 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:27.391209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1154/2325] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:27.393852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1172/2326] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:27.397438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd91b3-14b4-4e7e-abbb-b05d61d7c416 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:27.408957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd91b3-14b4-4e7e-abbb-b05d61d7c416 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:27.414234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ac0df-ec51-4925-82aa-99b15ba92df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:27.414837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ac0df-ec51-4925-82aa-99b15ba92df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.415150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ac0df-ec51-4925-82aa-99b15ba92df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.415418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ac0df-ec51-4925-82aa-99b15ba92df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.416765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ac0df-ec51-4925-82aa-99b15ba92df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:27.418905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ac0df-ec51-4925-82aa-99b15ba92df2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:27.520543 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1155/2327] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:27 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:27.521679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:27.532667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f35e6f-f212-470a-9ab1-77509fbe1b80 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:27.591961 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7b782-431c-4764-9aa0-1f1bcfc2ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:27.592490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7b782-431c-4764-9aa0-1f1bcfc2ffe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.592769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7b782-431c-4764-9aa0-1f1bcfc2ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.593042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7b782-431c-4764-9aa0-1f1bcfc2ffe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.593687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7b782-431c-4764-9aa0-1f1bcfc2ffe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:27.595676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7b782-431c-4764-9aa0-1f1bcfc2ffe2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:27.683057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1173/2328] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:27 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:27.694904 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ac0df-ec51-4925-82aa-99b15ba92df2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:27.712212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ac0df-ec51-4925-82aa-99b15ba92df2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:27.717316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbc5bd-95b0-4ec0-a503-c531eab90ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:27.723124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbc5bd-95b0-4ec0-a503-c531eab90ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.723124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbc5bd-95b0-4ec0-a503-c531eab90ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.723124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbc5bd-95b0-4ec0-a503-c531eab90ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.723124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbc5bd-95b0-4ec0-a503-c531eab90ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:27.723124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbc5bd-95b0-4ec0-a503-c531eab90ff3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:27.731589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1156/2329] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:27.735882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7b782-431c-4764-9aa0-1f1bcfc2ffe2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:27.751852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7b782-431c-4764-9aa0-1f1bcfc2ffe2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:27.765543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9029435-d412-42c6-9526-499e5ee7b3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:27.766216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029435-d412-42c6-9526-499e5ee7b3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.766486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029435-d412-42c6-9526-499e5ee7b3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.766754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029435-d412-42c6-9526-499e5ee7b3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.767458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9029435-d412-42c6-9526-499e5ee7b3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:27.781768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9029435-d412-42c6-9526-499e5ee7b3ad admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:27.781768 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1174/2330] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:27.786168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbc5bd-95b0-4ec0-a503-c531eab90ff3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:27.798447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbc5bd-95b0-4ec0-a503-c531eab90ff3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:27.805796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:27.806663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.806965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.807258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] PATH_INFO: `/v3/projects/dd8c8dfc28904b9199de355e61cf8678/users/090470d5e1964b71b8333e5acc82cd7c/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.831949 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8c8dfc28904b9199de355e61cf8678, user_id=090470d5e1964b71b8333e5acc82cd7c, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:27.844288 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:27.951187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1157/2331] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:27 2022] PUT /identity/v3/projects/dd8c8dfc28904b9199de355e61cf8678/users/090470d5e1964b71b8333e5acc82cd7c/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:27.963508 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50afe78-f176-42ed-8225-1898537cc30f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:27.963794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50afe78-f176-42ed-8225-1898537cc30f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:27.964062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50afe78-f176-42ed-8225-1898537cc30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:27.991123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e50afe78-f176-42ed-8225-1898537cc30f None None] MFA Rules not processed for user `090470d5e1964b71b8333e5acc82cd7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:27.992849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e50afe78-f176-42ed-8225-1898537cc30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:28.068331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1175/2332] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:28.080176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:28.081918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:28.140722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbc695-a154-490b-9cc5-6543cce00d4e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:28.142160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50afe78-f176-42ed-8225-1898537cc30f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:28.144681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb14b11-8162-4ea2-9535-953d7ac77468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:28.145500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb14b11-8162-4ea2-9535-953d7ac77468 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:28.145775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb14b11-8162-4ea2-9535-953d7ac77468 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:28.146043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb14b11-8162-4ea2-9535-953d7ac77468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:28.147919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5eb14b11-8162-4ea2-9535-953d7ac77468 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:28.151739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50afe78-f176-42ed-8225-1898537cc30f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:28.155457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed444eda-bc1d-4e0f-962a-60e38d721696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:28.156103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed444eda-bc1d-4e0f-962a-60e38d721696 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:28.156384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed444eda-bc1d-4e0f-962a-60e38d721696 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:28.160834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed444eda-bc1d-4e0f-962a-60e38d721696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:28.165665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed444eda-bc1d-4e0f-962a-60e38d721696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:28.167403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed444eda-bc1d-4e0f-962a-60e38d721696 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:28.176405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1176/2333] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:28.196597 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed444eda-bc1d-4e0f-962a-60e38d721696 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:28.213395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb14b11-8162-4ea2-9535-953d7ac77468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:28.215585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb14b11-8162-4ea2-9535-953d7ac77468 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:28.236767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1158/2334] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:28.262685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed444eda-bc1d-4e0f-962a-60e38d721696 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:28.267673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:28.268487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:28.268600 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:28.268804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] PATH_INFO: `/v3/users/090470d5e1964b71b8333e5acc82cd7c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:28.269949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=090470d5e1964b71b8333e5acc82cd7c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:28.272001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:28.453602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:28.536282 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090470d5e1964b71b8333e5acc82cd7c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:28.543902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090470d5e1964b71b8333e5acc82cd7c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:28.544593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090470d5e1964b71b8333e5acc82cd7c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:28.545202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090470d5e1964b71b8333e5acc82cd7c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:28.557534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090470d5e1964b71b8333e5acc82cd7c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:28.644656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1177/2335] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:28 2022] DELETE /identity/v3/users/090470d5e1964b71b8333e5acc82cd7c => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:28.650325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0016dc8-71f1-48db-9e4b-4e0a8043e0b7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:28.650691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0016dc8-71f1-48db-9e4b-4e0a8043e0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:28.651072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0016dc8-71f1-48db-9e4b-4e0a8043e0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:28.670372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b0016dc8-71f1-48db-9e4b-4e0a8043e0b7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:28.671604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b0016dc8-71f1-48db-9e4b-4e0a8043e0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:28.690108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:28.732436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1159/2336] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:28.744433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:28.745914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:28.805165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fef7c-4754-4c9a-8101-6d33ac5468b7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:28.808769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2f4b6-2061-4214-8283-6ef1a0a08442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:28.809368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2f4b6-2061-4214-8283-6ef1a0a08442 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:28.809644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2f4b6-2061-4214-8283-6ef1a0a08442 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:28.809912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2f4b6-2061-4214-8283-6ef1a0a08442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:28.818739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2f4b6-2061-4214-8283-6ef1a0a08442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:28.820216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2f4b6-2061-4214-8283-6ef1a0a08442 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:28.829808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1178/2337] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:29.395533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0016dc8-71f1-48db-9e4b-4e0a8043e0b7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:29.397190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b0016dc8-71f1-48db-9e4b-4e0a8043e0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:29.410182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0016dc8-71f1-48db-9e4b-4e0a8043e0b7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:29.415449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:29.416145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:29.416448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:29.416722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] PATH_INFO: `/v3/projects/dd8c8dfc28904b9199de355e61cf8678` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:29.417854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd8c8dfc28904b9199de355e61cf8678)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:29.419903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:29.558005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd8c8dfc28904b9199de355e61cf8678', 'request_id': 'req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:29.564273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd8c8dfc28904b9199de355e61cf8678', 'request_id': 'req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:29.571243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd8c8dfc28904b9199de355e61cf8678', 'request_id': 'req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:29.716777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] The token cache is being invalidate because project dd8c8dfc28904b9199de355e61cf8678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:29.717069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:29.717716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-dc72beae-60fb-49a3-a3c5-7ec3ef79d2d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:29.719627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1160/2338] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:29 2022] DELETE /identity/v3/projects/dd8c8dfc28904b9199de355e61cf8678 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:29.724890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7746263-8040-4d29-b17f-852b56d86022 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:29.725402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7746263-8040-4d29-b17f-852b56d86022 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:29.725499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7746263-8040-4d29-b17f-852b56d86022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:29.749265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b7746263-8040-4d29-b17f-852b56d86022 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:29.749265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b7746263-8040-4d29-b17f-852b56d86022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:29.818003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1179/2339] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:29.825841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9050-9067-4b1f-98e1-9ce51fa3778a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:29.826215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9050-9067-4b1f-98e1-9ce51fa3778a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:29.834061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9050-9067-4b1f-98e1-9ce51fa3778a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:29.846867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-04fc9050-9067-4b1f-98e1-9ce51fa3778a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:29.848096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-04fc9050-9067-4b1f-98e1-9ce51fa3778a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:29.856799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1161/2340] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:29.860921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7746263-8040-4d29-b17f-852b56d86022 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:29.871708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7746263-8040-4d29-b17f-852b56d86022 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:29.876362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:29.876974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:29.877255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:29.877519 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:29.878163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:29.878471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:29.880106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:29.893353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1180/2341] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:29.903670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc9050-9067-4b1f-98e1-9ce51fa3778a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:29.907654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc9050-9067-4b1f-98e1-9ce51fa3778a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:29.912554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da331b-0c54-46ef-973c-ff99516ada08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:29.913207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da331b-0c54-46ef-973c-ff99516ada08 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:29.914001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da331b-0c54-46ef-973c-ff99516ada08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:29.914281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da331b-0c54-46ef-973c-ff99516ada08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:29.914901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da331b-0c54-46ef-973c-ff99516ada08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:29.916834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da331b-0c54-46ef-973c-ff99516ada08 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:29.923955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:30.019947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:30.030098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1162/2342] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:29 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:30.033640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:30.044667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35490c5f-dab9-4411-ac80-1ed6fb05dd93 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:30.115226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793383c-fedd-4e2d-82a0-d1d7b1dd707d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:30.115886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793383c-fedd-4e2d-82a0-d1d7b1dd707d admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:30.116201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793383c-fedd-4e2d-82a0-d1d7b1dd707d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:30.116830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793383c-fedd-4e2d-82a0-d1d7b1dd707d admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:30.117625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793383c-fedd-4e2d-82a0-d1d7b1dd707d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:30.119880 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793383c-fedd-4e2d-82a0-d1d7b1dd707d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:30.305430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1181/2343] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:30 2022] POST /identity/v3/users => generated 292 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:30.309615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da331b-0c54-46ef-973c-ff99516ada08 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:30.326711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da331b-0c54-46ef-973c-ff99516ada08 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:30.334216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1763a4-957f-46fb-8f2b-4e3a7ddcf6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:30.334886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1763a4-957f-46fb-8f2b-4e3a7ddcf6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:30.337299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1763a4-957f-46fb-8f2b-4e3a7ddcf6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:30.337299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1763a4-957f-46fb-8f2b-4e3a7ddcf6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:30.337299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1763a4-957f-46fb-8f2b-4e3a7ddcf6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:30.338376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1763a4-957f-46fb-8f2b-4e3a7ddcf6bf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:30.349020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1163/2344] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:30.353070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793383c-fedd-4e2d-82a0-d1d7b1dd707d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:30.363703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793383c-fedd-4e2d-82a0-d1d7b1dd707d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:30.368428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca4def-83b1-4269-901a-e22a45af79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:30.369022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca4def-83b1-4269-901a-e22a45af79e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:30.369303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca4def-83b1-4269-901a-e22a45af79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:30.369574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca4def-83b1-4269-901a-e22a45af79e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:30.370213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca4def-83b1-4269-901a-e22a45af79e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:30.371922 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca4def-83b1-4269-901a-e22a45af79e5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:30.383345 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1182/2345] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:30.387240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1763a4-957f-46fb-8f2b-4e3a7ddcf6bf admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:30.397993 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1763a4-957f-46fb-8f2b-4e3a7ddcf6bf admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:30.402623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:30.403349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:30.403631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:30.403908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] PATH_INFO: `/v3/projects/2412df39492e4888b2dbddcd5397996e/users/2c816c9811524b3bbb27cf3af541cadf/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:30.419564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2412df39492e4888b2dbddcd5397996e, user_id=2c816c9811524b3bbb27cf3af541cadf, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:30.422795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:30.508027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1164/2346] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:30 2022] PUT /identity/v3/projects/2412df39492e4888b2dbddcd5397996e/users/2c816c9811524b3bbb27cf3af541cadf/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:30.518086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404ab4-22f7-4e40-ac77-708d4d916e36 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:30.518380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404ab4-22f7-4e40-ac77-708d4d916e36 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:30.518657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404ab4-22f7-4e40-ac77-708d4d916e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:30.548591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-43404ab4-22f7-4e40-ac77-708d4d916e36 None None] MFA Rules not processed for user `2c816c9811524b3bbb27cf3af541cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:30.549829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-43404ab4-22f7-4e40-ac77-708d4d916e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:30.593980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:30.597101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:30.624092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1183/2347] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:30.652335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43404ab4-22f7-4e40-ac77-708d4d916e36 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:30.654196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-43404ab4-22f7-4e40-ac77-708d4d916e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:30.674929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849afdc6-3c62-4e40-8710-141cedefc8bf admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:30.680360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:30.680985 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:30.681272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:30.681550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] PATH_INFO: `/v3/users/d56bcd7b11684160b8ab80eb9ea72195` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:30.682786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56bcd7b11684160b8ab80eb9ea72195)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:30.684750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:30.710152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43404ab4-22f7-4e40-ac77-708d4d916e36 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:30.713771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1ed25-3948-4c5a-8ffb-b719bc551dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:30.714369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ed25-3948-4c5a-8ffb-b719bc551dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:30.714647 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ed25-3948-4c5a-8ffb-b719bc551dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:30.714912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ed25-3948-4c5a-8ffb-b719bc551dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:30.723169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1ed25-3948-4c5a-8ffb-b719bc551dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:30.724582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1ed25-3948-4c5a-8ffb-b719bc551dd4 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:30.733165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1184/2348] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:30.921278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56bcd7b11684160b8ab80eb9ea72195'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:30.921991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56bcd7b11684160b8ab80eb9ea72195'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:30.934434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56bcd7b11684160b8ab80eb9ea72195'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:30.934434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56bcd7b11684160b8ab80eb9ea72195'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:31.043615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56bcd7b11684160b8ab80eb9ea72195'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:31.063564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1165/2349] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:30 2022] DELETE /identity/v3/users/d56bcd7b11684160b8ab80eb9ea72195 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:31.068057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2cbac-b295-44a7-b944-020ee0cf9d85 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:31.068446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2cbac-b295-44a7-b944-020ee0cf9d85 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:31.068789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2cbac-b295-44a7-b944-020ee0cf9d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:31.089541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2bb2cbac-b295-44a7-b944-020ee0cf9d85 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:31.090755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2bb2cbac-b295-44a7-b944-020ee0cf9d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:31.151512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1185/2350] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:31.161258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:31.163243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:31.213168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:31.230114 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f206fbc-e9df-4e23-9792-ec8096582c52 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:31.233757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:31.234374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:31.234656 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:31.235084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:31.245945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:31.247506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:31.258000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1166/2351] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:31.613281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:31.762129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2cbac-b295-44a7-b944-020ee0cf9d85 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:31.763700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2bb2cbac-b295-44a7-b944-020ee0cf9d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:31.776465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2cbac-b295-44a7-b944-020ee0cf9d85 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:31.783413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:31.784278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:31.784628 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:31.785363 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] PATH_INFO: `/v3/projects/ea123710d8664ef88bcb0a4b005f3c22` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:31.786630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea123710d8664ef88bcb0a4b005f3c22)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:31.794000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:31.951527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea123710d8664ef88bcb0a4b005f3c22', 'request_id': 'req-0da9a5f0-3839-4e42-afb1-49e6af0de030'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.008026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:32.010144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:32.023488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf94a7-7348-4654-a462-2e295b3e4e4d service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:32.046565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea123710d8664ef88bcb0a4b005f3c22', 'request_id': 'req-0da9a5f0-3839-4e42-afb1-49e6af0de030'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.054771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea123710d8664ef88bcb0a4b005f3c22', 'request_id': 'req-0da9a5f0-3839-4e42-afb1-49e6af0de030'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.074157 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] The token cache is being invalidate because project ea123710d8664ef88bcb0a4b005f3c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:32.074608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.075312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0da9a5f0-3839-4e42-afb1-49e6af0de030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.079211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1186/2352] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:31 2022] DELETE /identity/v3/projects/ea123710d8664ef88bcb0a4b005f3c22 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:32.082788 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b77e-2747-4e0b-9fef-32b650a4c361 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.083305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b77e-2747-4e0b-9fef-32b650a4c361 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.083630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5b77e-2747-4e0b-9fef-32b650a4c361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.096133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:32.096133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.096133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.096133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] PATH_INFO: `/v3/users/0ac3c8a923a24a99ad680cec81d8eb23` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.096761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac3c8a923a24a99ad680cec81d8eb23)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:32.096761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:32.106746 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-02f5b77e-2747-4e0b-9fef-32b650a4c361 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:32.108228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-02f5b77e-2747-4e0b-9fef-32b650a4c361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:32.118480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1187/2353] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:32.126578 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658716a-2480-41b5-a3f0-09e7fcebc31c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.126861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658716a-2480-41b5-a3f0-09e7fcebc31c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.127187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658716a-2480-41b5-a3f0-09e7fcebc31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.147967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5658716a-2480-41b5-a3f0-09e7fcebc31c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:32.149210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5658716a-2480-41b5-a3f0-09e7fcebc31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:32.157648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1188/2354] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:32.161790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658716a-2480-41b5-a3f0-09e7fcebc31c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:32.171093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658716a-2480-41b5-a3f0-09e7fcebc31c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:32.175819 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:32.179264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.179264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.179264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.179264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:32.179264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:32.180254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:32.191703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1189/2355] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:19:32.196545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:32.206388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53502b00-bb2a-4a18-ac22-52ede57b93f2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:32.211355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0f90-33dd-4d85-aacc-792b7b6ff491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:32.215247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0f90-33dd-4d85-aacc-792b7b6ff491 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.215247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0f90-33dd-4d85-aacc-792b7b6ff491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.215247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0f90-33dd-4d85-aacc-792b7b6ff491 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.215247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f0f90-33dd-4d85-aacc-792b7b6ff491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:32.215247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f0f90-33dd-4d85-aacc-792b7b6ff491 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:32.331556 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1190/2356] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:32 2022] POST /identity/v3/projects => generated 397 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:32.332932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0f90-33dd-4d85-aacc-792b7b6ff491 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:32.335273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac3c8a923a24a99ad680cec81d8eb23'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.336031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac3c8a923a24a99ad680cec81d8eb23'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.342990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac3c8a923a24a99ad680cec81d8eb23'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.343654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac3c8a923a24a99ad680cec81d8eb23'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.394840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0f90-33dd-4d85-aacc-792b7b6ff491 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:32.399734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:32.400445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.400629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.400924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.401502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:32.403485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:32.463759 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-42dd62d1-5de9-4644-9b74-e3d381866107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac3c8a923a24a99ad680cec81d8eb23'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:32.481430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1167/2357] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:32 2022] DELETE /identity/v3/users/0ac3c8a923a24a99ad680cec81d8eb23 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:32.483425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9935c-3714-4550-aebd-9e483b25fd38 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.483711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9935c-3714-4550-aebd-9e483b25fd38 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.483988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9935c-3714-4550-aebd-9e483b25fd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.503061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6bb9935c-3714-4550-aebd-9e483b25fd38 None None] MFA Rules not processed for user `45777028be184247b0a58eee7d2c7e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:32.504254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6bb9935c-3714-4550-aebd-9e483b25fd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:32.558246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1168/2358] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:32.559494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23464-1a1a-4a4c-9559-3a2ac3db3b94 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.559773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23464-1a1a-4a4c-9559-3a2ac3db3b94 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.560045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea23464-1a1a-4a4c-9559-3a2ac3db3b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.578751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5ea23464-1a1a-4a4c-9559-3a2ac3db3b94 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:32.580002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5ea23464-1a1a-4a4c-9559-3a2ac3db3b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:32.603603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1191/2359] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:32 2022] POST /identity/v3/users => generated 299 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:32.607451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:32.609288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:32.631213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d31fc-e37f-4e03-ac37-46da4c2a6bf9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:32.644963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1169/2360] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:32.655052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea23464-1a1a-4a4c-9559-3a2ac3db3b94 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:32.656492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5ea23464-1a1a-4a4c-9559-3a2ac3db3b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:32.707449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1080d3-5123-4336-a920-2af1c1ceeb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:32.708340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1080d3-5123-4336-a920-2af1c1ceeb72 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.708625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1080d3-5123-4336-a920-2af1c1ceeb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.708894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1080d3-5123-4336-a920-2af1c1ceeb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.709576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1080d3-5123-4336-a920-2af1c1ceeb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:32.711433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1080d3-5123-4336-a920-2af1c1ceeb72 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:32.720903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea23464-1a1a-4a4c-9559-3a2ac3db3b94 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:32.722935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:32.726297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7fca4-8d07-4171-b29e-a297e5eec8e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:32.730322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7fca4-8d07-4171-b29e-a297e5eec8e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.730322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7fca4-8d07-4171-b29e-a297e5eec8e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.730322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7fca4-8d07-4171-b29e-a297e5eec8e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.730589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1192/2361] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:32.733172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1080d3-5123-4336-a920-2af1c1ceeb72 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:32.745502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1080d3-5123-4336-a920-2af1c1ceeb72 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:32.747521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7fca4-8d07-4171-b29e-a297e5eec8e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:32.749287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7fca4-8d07-4171-b29e-a297e5eec8e5 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:32.751701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78674f6-5be4-49a3-a009-d424cbda86ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:32.752761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78674f6-5be4-49a3-a009-d424cbda86ad admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.753037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78674f6-5be4-49a3-a009-d424cbda86ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.753308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78674f6-5be4-49a3-a009-d424cbda86ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.754342 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78674f6-5be4-49a3-a009-d424cbda86ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:32.760163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78674f6-5be4-49a3-a009-d424cbda86ad admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:32.763720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1170/2362] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:32.775170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1193/2363] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:32.777680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7fca4-8d07-4171-b29e-a297e5eec8e5 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:32.788966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7fca4-8d07-4171-b29e-a297e5eec8e5 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:32.794156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:32.794831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.795162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.800798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] PATH_INFO: `/v3/projects/caf798435cb8497b9b70f941d04c3530/users/975a080756254318a4435e274d41bdba/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.813670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf798435cb8497b9b70f941d04c3530, user_id=975a080756254318a4435e274d41bdba, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:32.816579 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:32.929299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1171/2364] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:32 2022] PUT /identity/v3/projects/caf798435cb8497b9b70f941d04c3530/users/975a080756254318a4435e274d41bdba/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:32.940575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279aa41a-3b74-421f-91bb-b3e0b8cb64af None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:32.940932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279aa41a-3b74-421f-91bb-b3e0b8cb64af None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:32.941322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279aa41a-3b74-421f-91bb-b3e0b8cb64af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:32.971201 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-279aa41a-3b74-421f-91bb-b3e0b8cb64af None None] MFA Rules not processed for user `975a080756254318a4435e274d41bdba`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:32.972573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-279aa41a-3b74-421f-91bb-b3e0b8cb64af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:33.038561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:33.083303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1194/2365] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:33.096991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:33.106451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:33.177939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a654b7-e8b1-4cf6-b76c-840e96c29fd6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:33.182623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866566a-5179-458a-a7c6-104fe2478a84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:33.183380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866566a-5179-458a-a7c6-104fe2478a84 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:33.183731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866566a-5179-458a-a7c6-104fe2478a84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:33.184071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866566a-5179-458a-a7c6-104fe2478a84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:33.196935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866566a-5179-458a-a7c6-104fe2478a84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:33.198548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866566a-5179-458a-a7c6-104fe2478a84 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:33.210438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1172/2366] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:33.314644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279aa41a-3b74-421f-91bb-b3e0b8cb64af None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:33.316299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-279aa41a-3b74-421f-91bb-b3e0b8cb64af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:33.421203 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279aa41a-3b74-421f-91bb-b3e0b8cb64af None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:33.427150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:33.428521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:33.429036 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:33.429524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] PATH_INFO: `/v3/projects/f5d0ee0087dd4032b371164ebef2973b` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:33.431059 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d0ee0087dd4032b371164ebef2973b)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:33.434424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:33.577284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d0ee0087dd4032b371164ebef2973b', 'request_id': 'req-103d1125-3d9b-443d-ab5b-bb69e934fd21'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:33.640470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d0ee0087dd4032b371164ebef2973b', 'request_id': 'req-103d1125-3d9b-443d-ab5b-bb69e934fd21'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:33.646330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d0ee0087dd4032b371164ebef2973b', 'request_id': 'req-103d1125-3d9b-443d-ab5b-bb69e934fd21'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:33.658025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] The token cache is being invalidate because project f5d0ee0087dd4032b371164ebef2973b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:33.658314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:33.660014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-103d1125-3d9b-443d-ab5b-bb69e934fd21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:33.665634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1195/2367] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:33 2022] DELETE /identity/v3/projects/f5d0ee0087dd4032b371164ebef2973b => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:33.669013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424e4b-e1be-44a1-b7c7-73d1d888191b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:33.669013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424e4b-e1be-44a1-b7c7-73d1d888191b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:33.669013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424e4b-e1be-44a1-b7c7-73d1d888191b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:33.697043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1b424e4b-e1be-44a1-b7c7-73d1d888191b None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:33.698642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1b424e4b-e1be-44a1-b7c7-73d1d888191b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:33.764409 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1173/2368] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:33.772141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42badc6-9df4-4d95-bab3-cdf5ce3d71bd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:33.772461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42badc6-9df4-4d95-bab3-cdf5ce3d71bd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:33.772739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42badc6-9df4-4d95-bab3-cdf5ce3d71bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:33.798670 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b42badc6-9df4-4d95-bab3-cdf5ce3d71bd None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:33.799921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b42badc6-9df4-4d95-bab3-cdf5ce3d71bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:33.800543 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:33.808651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1196/2369] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:33.813242 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b424e4b-e1be-44a1-b7c7-73d1d888191b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:33.823765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b424e4b-e1be-44a1-b7c7-73d1d888191b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:33.843497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:33.844194 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:33.844478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:33.846070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:33.846070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:33.846070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:33.847914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:33.860299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1174/2370] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:33.864904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42badc6-9df4-4d95-bab3-cdf5ce3d71bd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:33.874647 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42badc6-9df4-4d95-bab3-cdf5ce3d71bd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:33.879598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d049-3f68-413d-b04c-92fa1067b47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:33.880322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d049-3f68-413d-b04c-92fa1067b47b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:33.880599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d049-3f68-413d-b04c-92fa1067b47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:33.881461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d049-3f68-413d-b04c-92fa1067b47b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:33.882084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1d049-3f68-413d-b04c-92fa1067b47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:33.884044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1d049-3f68-413d-b04c-92fa1067b47b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:33.908689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:33.967128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1197/2371] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:33 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:33.971468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:33.981908 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7b2a0-046a-4ebd-9818-bfb7e9f57596 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:34.037705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1850d7-e452-43fc-b64e-8a65f45688ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:34.038299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1850d7-e452-43fc-b64e-8a65f45688ed admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:34.038576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1850d7-e452-43fc-b64e-8a65f45688ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:34.038844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1850d7-e452-43fc-b64e-8a65f45688ed admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:34.039491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1850d7-e452-43fc-b64e-8a65f45688ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:34.043545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1850d7-e452-43fc-b64e-8a65f45688ed admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:34.162890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1175/2372] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:33 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:34.166846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d049-3f68-413d-b04c-92fa1067b47b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:34.176968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d049-3f68-413d-b04c-92fa1067b47b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:34.182638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4302c-630f-4fa8-88de-fdeb9dc2f867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:34.185640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4302c-630f-4fa8-88de-fdeb9dc2f867 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:34.185640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4302c-630f-4fa8-88de-fdeb9dc2f867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:34.185640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4302c-630f-4fa8-88de-fdeb9dc2f867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:34.185640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4302c-630f-4fa8-88de-fdeb9dc2f867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:34.186784 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4302c-630f-4fa8-88de-fdeb9dc2f867 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:34.197067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1198/2373] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:34.201390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1850d7-e452-43fc-b64e-8a65f45688ed admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:34.211833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1850d7-e452-43fc-b64e-8a65f45688ed admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:34.216970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbee835-1ceb-48a3-801b-c255da6b8838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:34.217566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbee835-1ceb-48a3-801b-c255da6b8838 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:34.217844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbee835-1ceb-48a3-801b-c255da6b8838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:34.218115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbee835-1ceb-48a3-801b-c255da6b8838 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:34.218749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbee835-1ceb-48a3-801b-c255da6b8838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:34.220396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbee835-1ceb-48a3-801b-c255da6b8838 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:34.233432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1176/2374] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:34.234887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:34.237553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4302c-630f-4fa8-88de-fdeb9dc2f867 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:34.248580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4302c-630f-4fa8-88de-fdeb9dc2f867 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:34.265082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:34.265745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:34.266030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:34.266459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] PATH_INFO: `/v3/projects/333a97c605cc42ea8865885bfdaecee1/users/515ff673dc394aceae078cb52ca607dd/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:34.281911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333a97c605cc42ea8865885bfdaecee1, user_id=515ff673dc394aceae078cb52ca607dd, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:34.284656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:34.364816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1199/2375] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:34 2022] PUT /identity/v3/projects/333a97c605cc42ea8865885bfdaecee1/users/515ff673dc394aceae078cb52ca607dd/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:34.377164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf98ed-35e4-4e11-a4bb-38970f38aa9a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:34.377714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf98ed-35e4-4e11-a4bb-38970f38aa9a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:34.377714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf98ed-35e4-4e11-a4bb-38970f38aa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:34.417734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e8bf98ed-35e4-4e11-a4bb-38970f38aa9a None None] MFA Rules not processed for user `515ff673dc394aceae078cb52ca607dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:34.419385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e8bf98ed-35e4-4e11-a4bb-38970f38aa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:34.493027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1177/2376] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:34.504749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:34.506557 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:34.668521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc1ea6-c455-4503-b348-a21c612f9050 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:34.677125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9890260-bb54-4885-82b3-54a339be8fcb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e9526306889402c83f2d33b340c2dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:34.677125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9890260-bb54-4885-82b3-54a339be8fcb service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:34.677125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9890260-bb54-4885-82b3-54a339be8fcb service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:34.677125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9890260-bb54-4885-82b3-54a339be8fcb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:34.691275 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9890260-bb54-4885-82b3-54a339be8fcb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:34.693048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9890260-bb54-4885-82b3-54a339be8fcb service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:34.706317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1200/2377] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:35.048706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:35.314058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:35.639661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:35.934040 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:36.355795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:36.526519 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:36.590839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:36.810745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:36.900935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:37.133653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:37.162436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:37.281892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:37.382224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:37.750650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:37.776270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:37.925182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:37.943135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:38.218820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:38.290155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:38.370130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:38.506031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:40.656175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf98ed-35e4-4e11-a4bb-38970f38aa9a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:40.659683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e8bf98ed-35e4-4e11-a4bb-38970f38aa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:40.748629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf98ed-35e4-4e11-a4bb-38970f38aa9a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:40.753252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:40.754012 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:40.754284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:40.756403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] PATH_INFO: `/v3/users/c64637d2aa4b43a68b89908074eb1375` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:40.756403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64637d2aa4b43a68b89908074eb1375)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:40.758117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:40.960988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64637d2aa4b43a68b89908074eb1375'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:40.961658 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64637d2aa4b43a68b89908074eb1375'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:40.969814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64637d2aa4b43a68b89908074eb1375'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:40.970569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64637d2aa4b43a68b89908074eb1375'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:41.049687 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64637d2aa4b43a68b89908074eb1375'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:41.073605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1178/2378] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:40 2022] DELETE /identity/v3/users/c64637d2aa4b43a68b89908074eb1375 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:41.078162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb5bd3-2177-4185-980b-9fe3eb661045 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:41.078763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb5bd3-2177-4185-980b-9fe3eb661045 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:41.078763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb5bd3-2177-4185-980b-9fe3eb661045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:41.125143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a4cb5bd3-2177-4185-980b-9fe3eb661045 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:41.126633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a4cb5bd3-2177-4185-980b-9fe3eb661045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:41.154469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:41.218855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1201/2379] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:41.234390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:41.239632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:41.276780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:41.318641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662bf89-db9f-4e7a-96a1-99f4bf553a44 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:41.323232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b94fc7-3a0c-4603-bb4d-212f54263194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:41.324215 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b94fc7-3a0c-4603-bb4d-212f54263194 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:41.324598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b94fc7-3a0c-4603-bb4d-212f54263194 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:41.324931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b94fc7-3a0c-4603-bb4d-212f54263194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:41.343207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b94fc7-3a0c-4603-bb4d-212f54263194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:41.344830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b94fc7-3a0c-4603-bb4d-212f54263194 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:41.355100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1179/2380] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:41.825789 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb5bd3-2177-4185-980b-9fe3eb661045 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:41.827271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a4cb5bd3-2177-4185-980b-9fe3eb661045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:41.840101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb5bd3-2177-4185-980b-9fe3eb661045 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:41.844795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:41.845423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:41.850823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:41.850823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] PATH_INFO: `/v3/projects/57bec21bdcc0435f9cfd1c935bc66cab` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:41.850823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=57bec21bdcc0435f9cfd1c935bc66cab)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:41.850823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:41.981867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57bec21bdcc0435f9cfd1c935bc66cab', 'request_id': 'req-417f394e-2e02-42b6-a137-885833d1cb3c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.034638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57bec21bdcc0435f9cfd1c935bc66cab', 'request_id': 'req-417f394e-2e02-42b6-a137-885833d1cb3c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.041118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57bec21bdcc0435f9cfd1c935bc66cab', 'request_id': 'req-417f394e-2e02-42b6-a137-885833d1cb3c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.052263 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] The token cache is being invalidate because project 57bec21bdcc0435f9cfd1c935bc66cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:42.060615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.060615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.060615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1202/2381] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:41 2022] DELETE /identity/v3/projects/57bec21bdcc0435f9cfd1c935bc66cab => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:42.060900 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b94fc7-3a0c-4603-bb4d-212f54263194 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:42.068510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b94fc7-3a0c-4603-bb4d-212f54263194 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:42.126758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:42.127413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:42.127739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:42.128016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] PATH_INFO: `/v3/users/2ee78389a9d04f3a8396d3b8d05637c9` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:42.129187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee78389a9d04f3a8396d3b8d05637c9)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:42.131138 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:42.319378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee78389a9d04f3a8396d3b8d05637c9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.320354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee78389a9d04f3a8396d3b8d05637c9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.329394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee78389a9d04f3a8396d3b8d05637c9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.330424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee78389a9d04f3a8396d3b8d05637c9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.412350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee78389a9d04f3a8396d3b8d05637c9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:42.436245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1180/2382] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:42 2022] DELETE /identity/v3/users/2ee78389a9d04f3a8396d3b8d05637c9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:43.027850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:43.029311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:43.043416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f394e-2e02-42b6-a137-885833d1cb3c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:43.101061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:43.101682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:43.101984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:43.102254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] PATH_INFO: `/v3/projects/247e95e31f5c4ad9ab436710394c1f5f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:43.103422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] RBAC: Authorizing `identity:delete_project(project_id=247e95e31f5c4ad9ab436710394c1f5f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:43.105326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:43.248800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:43.250017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247e95e31f5c4ad9ab436710394c1f5f', 'request_id': 'req-0e255ac1-82ee-4ae7-9004-a63eb58b447a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.358073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247e95e31f5c4ad9ab436710394c1f5f', 'request_id': 'req-0e255ac1-82ee-4ae7-9004-a63eb58b447a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.365103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247e95e31f5c4ad9ab436710394c1f5f', 'request_id': 'req-0e255ac1-82ee-4ae7-9004-a63eb58b447a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.376320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] The token cache is being invalidate because project 247e95e31f5c4ad9ab436710394c1f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:43.376603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.377234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.378741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1203/2383] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:43 2022] DELETE /identity/v3/projects/247e95e31f5c4ad9ab436710394c1f5f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:43.382861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:43.384326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:43.397812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09671b32-a7f4-4611-86f6-c9d9e0dffb8e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:43.446305 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:43.461488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:43.462113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:43.462510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:43.462876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] PATH_INFO: `/v3/users/474f138355df4f11a2f7faee97b7ae4f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:43.464031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=474f138355df4f11a2f7faee97b7ae4f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:43.465939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:43.734019 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474f138355df4f11a2f7faee97b7ae4f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.734669 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474f138355df4f11a2f7faee97b7ae4f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.741818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474f138355df4f11a2f7faee97b7ae4f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.742481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474f138355df4f11a2f7faee97b7ae4f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.814727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d26703e3-f73d-4746-b5eb-ff9a1f3bc731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474f138355df4f11a2f7faee97b7ae4f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:43.834064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1181/2384] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:43 2022] DELETE /identity/v3/users/474f138355df4f11a2f7faee97b7ae4f => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:44.256394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:44.257858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:44.275743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255ac1-82ee-4ae7-9004-a63eb58b447a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:44.334992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:44.335813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:44.335931 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:44.336161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] PATH_INFO: `/v3/projects/fa026f44e9714c978737af132a7674c3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:44.337293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa026f44e9714c978737af132a7674c3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:44.339271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:44.454570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa026f44e9714c978737af132a7674c3', 'request_id': 'req-78099d62-978a-41dc-b80b-51309a11c924'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:44.565694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa026f44e9714c978737af132a7674c3', 'request_id': 'req-78099d62-978a-41dc-b80b-51309a11c924'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:44.571831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa026f44e9714c978737af132a7674c3', 'request_id': 'req-78099d62-978a-41dc-b80b-51309a11c924'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:44.583400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] The token cache is being invalidate because project fa026f44e9714c978737af132a7674c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:44.583691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:44.584324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-78099d62-978a-41dc-b80b-51309a11c924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:44.585874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1204/2385] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:44 2022] DELETE /identity/v3/projects/fa026f44e9714c978737af132a7674c3 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:44.591499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54068514-a7ae-4bb0-831b-a035f0c8ab10 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:44.591947 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54068514-a7ae-4bb0-831b-a035f0c8ab10 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:44.592091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54068514-a7ae-4bb0-831b-a035f0c8ab10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:44.611825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-54068514-a7ae-4bb0-831b-a035f0c8ab10 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:44.613637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-54068514-a7ae-4bb0-831b-a035f0c8ab10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:44.672109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1182/2386] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:44.682697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c751ba-7c66-49b9-9773-eed56672484c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:44.683021 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c751ba-7c66-49b9-9773-eed56672484c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:44.683302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c751ba-7c66-49b9-9773-eed56672484c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:44.703122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-44c751ba-7c66-49b9-9773-eed56672484c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:44.704589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-44c751ba-7c66-49b9-9773-eed56672484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:44.712665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1205/2387] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:44.720586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54068514-a7ae-4bb0-831b-a035f0c8ab10 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:44.732006 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54068514-a7ae-4bb0-831b-a035f0c8ab10 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:44.739301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:44.751995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:44.751995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:44.751995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:44.751995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:44.751995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:44.751995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:44.761559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1183/2388] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:44.761830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c751ba-7c66-49b9-9773-eed56672484c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:44.772431 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c751ba-7c66-49b9-9773-eed56672484c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:44.779634 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d79b07-cb9b-4eae-a9eb-fe55b89f1a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:44.780413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d79b07-cb9b-4eae-a9eb-fe55b89f1a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:44.780837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d79b07-cb9b-4eae-a9eb-fe55b89f1a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:44.781271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d79b07-cb9b-4eae-a9eb-fe55b89f1a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:44.782116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d79b07-cb9b-4eae-a9eb-fe55b89f1a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:44.786053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d79b07-cb9b-4eae-a9eb-fe55b89f1a9c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:44.880698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1206/2389] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:44 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:44.886965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:44.897913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee27034-caf6-4001-bdd5-71ce30a2fbf5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:44.949942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:44.960087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cabd6-5d43-4b68-a085-bb328a6e599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:44.961597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cabd6-5d43-4b68-a085-bb328a6e599a admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:44.961597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cabd6-5d43-4b68-a085-bb328a6e599a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:44.961597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cabd6-5d43-4b68-a085-bb328a6e599a admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:44.961858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cabd6-5d43-4b68-a085-bb328a6e599a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:44.963894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cabd6-5d43-4b68-a085-bb328a6e599a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:45.042882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1184/2390] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:44 2022] POST /identity/v3/users => generated 284 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:45.051869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d79b07-cb9b-4eae-a9eb-fe55b89f1a9c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:45.062236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d79b07-cb9b-4eae-a9eb-fe55b89f1a9c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:45.067801 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d62786-d4fc-4be4-b0e2-624381fa6ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:45.068395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62786-d4fc-4be4-b0e2-624381fa6ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:45.068668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62786-d4fc-4be4-b0e2-624381fa6ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:45.068934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62786-d4fc-4be4-b0e2-624381fa6ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:45.069615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d62786-d4fc-4be4-b0e2-624381fa6ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:45.071249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d62786-d4fc-4be4-b0e2-624381fa6ba3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:45.082412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1207/2391] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:45.086443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cabd6-5d43-4b68-a085-bb328a6e599a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:45.097709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cabd6-5d43-4b68-a085-bb328a6e599a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:45.102449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f82e5-9978-4f2d-8cde-8bbcc1c523dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:45.103081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f82e5-9978-4f2d-8cde-8bbcc1c523dd admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:45.103365 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f82e5-9978-4f2d-8cde-8bbcc1c523dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:45.103640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f82e5-9978-4f2d-8cde-8bbcc1c523dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:45.104378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f82e5-9978-4f2d-8cde-8bbcc1c523dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:45.105924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f82e5-9978-4f2d-8cde-8bbcc1c523dd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:45.117340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1185/2392] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:45.121546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d62786-d4fc-4be4-b0e2-624381fa6ba3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:45.133973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d62786-d4fc-4be4-b0e2-624381fa6ba3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:45.139576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:45.140490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:45.140892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:45.141331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] PATH_INFO: `/v3/projects/0db1595fa58d47f7ab3ae158b29a0e49/users/adfe865874114e618d1f208ffa685762/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:45.158222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db1595fa58d47f7ab3ae158b29a0e49, user_id=adfe865874114e618d1f208ffa685762, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:45.161711 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:45.228073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1208/2393] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:45 2022] PUT /identity/v3/projects/0db1595fa58d47f7ab3ae158b29a0e49/users/adfe865874114e618d1f208ffa685762/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:45.238725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb6327-fdad-4fd7-8eee-6acee3dc0491 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:45.239040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb6327-fdad-4fd7-8eee-6acee3dc0491 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:45.239312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb6327-fdad-4fd7-8eee-6acee3dc0491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:45.267437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-20eb6327-fdad-4fd7-8eee-6acee3dc0491 None None] MFA Rules not processed for user `adfe865874114e618d1f208ffa685762`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:45.268650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-20eb6327-fdad-4fd7-8eee-6acee3dc0491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:45.332329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1186/2394] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:45.342349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:45.344191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:45.350057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:45.403561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fcf33-47ca-4acf-8b18-90cca8d6f6d0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:45.407303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bfa6e-e491-4165-9fa1-42dded030e0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e9526306889402c83f2d33b340c2dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:45.408149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bfa6e-e491-4165-9fa1-42dded030e0e service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:45.408658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bfa6e-e491-4165-9fa1-42dded030e0e service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:45.409852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bfa6e-e491-4165-9fa1-42dded030e0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:45.418313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bfa6e-e491-4165-9fa1-42dded030e0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:45.419750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bfa6e-e491-4165-9fa1-42dded030e0e service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:45.429163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1209/2395] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:48.166521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb6327-fdad-4fd7-8eee-6acee3dc0491 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:48.173206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-20eb6327-fdad-4fd7-8eee-6acee3dc0491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:48.243227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb6327-fdad-4fd7-8eee-6acee3dc0491 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:48.247776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:48.248399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:48.248698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:48.248969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] PATH_INFO: `/v3/users/2c816c9811524b3bbb27cf3af541cadf` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:48.250084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c816c9811524b3bbb27cf3af541cadf)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:48.252783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:48.660426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c816c9811524b3bbb27cf3af541cadf'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:48.661170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c816c9811524b3bbb27cf3af541cadf'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:48.670915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c816c9811524b3bbb27cf3af541cadf'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:48.671981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c816c9811524b3bbb27cf3af541cadf'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:48.679103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:48.745679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c816c9811524b3bbb27cf3af541cadf'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:48.767069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1187/2396] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:48 2022] DELETE /identity/v3/users/2c816c9811524b3bbb27cf3af541cadf => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:48.772228 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b968ec-742e-48ae-bfb4-47c1a059d568 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:48.780245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b968ec-742e-48ae-bfb4-47c1a059d568 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:48.780245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b968ec-742e-48ae-bfb4-47c1a059d568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:48.793428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-98b968ec-742e-48ae-bfb4-47c1a059d568 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:48.794655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-98b968ec-742e-48ae-bfb4-47c1a059d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:48.853045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1210/2397] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:48.862790 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:48.864263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:48.921723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5e36f-9e13-4ff2-9d0b-51faa8775a34 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:48.925377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:48.925974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:48.926254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:48.926526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:48.935535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:48.936911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:48.946440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1188/2398] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:49.441693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b968ec-742e-48ae-bfb4-47c1a059d568 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:49.443437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-98b968ec-742e-48ae-bfb4-47c1a059d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:49.458756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b968ec-742e-48ae-bfb4-47c1a059d568 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:49.474694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:49.475587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:49.475904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:49.476256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] PATH_INFO: `/v3/projects/2412df39492e4888b2dbddcd5397996e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:49.477484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2412df39492e4888b2dbddcd5397996e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:49.480101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:49.633856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2412df39492e4888b2dbddcd5397996e', 'request_id': 'req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:49.671262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:49.675440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:49.686055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0ce71-a4c1-4bc5-9f26-bcccc55274e1 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:49.727925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2412df39492e4888b2dbddcd5397996e', 'request_id': 'req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:49.740349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2412df39492e4888b2dbddcd5397996e', 'request_id': 'req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:49.753905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:49.754547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:49.754825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:49.755543 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] PATH_INFO: `/v3/users/a53baa179e3a4a7eb06f088783d014fb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:49.757808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53baa179e3a4a7eb06f088783d014fb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:49.760835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:49.764694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] The token cache is being invalidate because project 2412df39492e4888b2dbddcd5397996e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:49.764993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:49.765705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6fb4a7e6-ed3f-47b7-9e65-a5d974214f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:49.767271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1211/2399] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:49 2022] DELETE /identity/v3/projects/2412df39492e4888b2dbddcd5397996e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:49.771959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326ccb-f39b-49d9-bc25-dea7e2fca38f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:49.773136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326ccb-f39b-49d9-bc25-dea7e2fca38f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:49.773136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18326ccb-f39b-49d9-bc25-dea7e2fca38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:49.812130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-18326ccb-f39b-49d9-bc25-dea7e2fca38f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:49.813393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-18326ccb-f39b-49d9-bc25-dea7e2fca38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:49.821473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1212/2400] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:49.830883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277eafa-50a8-4c40-9ea9-a0ab46e0d4a3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:49.831193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277eafa-50a8-4c40-9ea9-a0ab46e0d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:49.831464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277eafa-50a8-4c40-9ea9-a0ab46e0d4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:49.854184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7277eafa-50a8-4c40-9ea9-a0ab46e0d4a3 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:49.855777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7277eafa-50a8-4c40-9ea9-a0ab46e0d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:49.865565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1213/2401] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:49.869436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277eafa-50a8-4c40-9ea9-a0ab46e0d4a3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:49.878715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277eafa-50a8-4c40-9ea9-a0ab46e0d4a3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:49.883325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:49.883929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:49.884208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:49.884485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:49.885165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:49.885423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:49.887047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:49.898995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1214/2402] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:19:49.903399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:49.910806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:49.913906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7caaf4-b138-4779-90b7-e49504fc259c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:49.918567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:49.919169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:49.919449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:49.919723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:49.920333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:49.922197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:50.021103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53baa179e3a4a7eb06f088783d014fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.021709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53baa179e3a4a7eb06f088783d014fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.028774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53baa179e3a4a7eb06f088783d014fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.029424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53baa179e3a4a7eb06f088783d014fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.044324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1215/2403] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:49 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:50.049152 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:50.050710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:50.118580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be640324-0b4f-4654-a19f-21011f57c8cd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:50.118965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-02980402-a917-48af-add1-2269e4694b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53baa179e3a4a7eb06f088783d014fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.123839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192525e-295f-4f3a-bc4f-9d3a6f6b2199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:50.124474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192525e-295f-4f3a-bc4f-9d3a6f6b2199 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.124757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192525e-295f-4f3a-bc4f-9d3a6f6b2199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.125145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192525e-295f-4f3a-bc4f-9d3a6f6b2199 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.125838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192525e-295f-4f3a-bc4f-9d3a6f6b2199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:50.127830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192525e-295f-4f3a-bc4f-9d3a6f6b2199 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:50.134024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:50.138326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1189/2404] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:49 2022] DELETE /identity/v3/users/a53baa179e3a4a7eb06f088783d014fb => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:50.143205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995aa24-fbfd-4142-9897-f5cce24a0d69 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.143496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995aa24-fbfd-4142-9897-f5cce24a0d69 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.143776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995aa24-fbfd-4142-9897-f5cce24a0d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.162965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e995aa24-fbfd-4142-9897-f5cce24a0d69 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:50.165906 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e995aa24-fbfd-4142-9897-f5cce24a0d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:50.229615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1190/2405] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:19:50.235805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995aa24-fbfd-4142-9897-f5cce24a0d69 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:50.237234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e995aa24-fbfd-4142-9897-f5cce24a0d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:50.295375 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1216/2406] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:50 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:50.296309 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192525e-295f-4f3a-bc4f-9d3a6f6b2199 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:50.299492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995aa24-fbfd-4142-9897-f5cce24a0d69 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:50.304025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49983-a4d5-41ca-9924-45bddee0f364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:50.304677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49983-a4d5-41ca-9924-45bddee0f364 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.304967 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49983-a4d5-41ca-9924-45bddee0f364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.305245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49983-a4d5-41ca-9924-45bddee0f364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.305999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192525e-295f-4f3a-bc4f-9d3a6f6b2199 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:50.310622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf36a-08c4-4998-b063-c87d57e6d8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:50.311242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf36a-08c4-4998-b063-c87d57e6d8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.311521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf36a-08c4-4998-b063-c87d57e6d8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.311794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf36a-08c4-4998-b063-c87d57e6d8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.312426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cf36a-08c4-4998-b063-c87d57e6d8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:50.314065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cf36a-08c4-4998-b063-c87d57e6d8d1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:50.314922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b49983-a4d5-41ca-9924-45bddee0f364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:50.320352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b49983-a4d5-41ca-9924-45bddee0f364 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:50.327756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1217/2407] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:50.328932 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf36a-08c4-4998-b063-c87d57e6d8d1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:50.332355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1191/2408] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:50.339031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf36a-08c4-4998-b063-c87d57e6d8d1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:50.344255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f155d98-0a52-40cd-b939-92f6ef167de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:50.344847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f155d98-0a52-40cd-b939-92f6ef167de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.345150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f155d98-0a52-40cd-b939-92f6ef167de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.346001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f155d98-0a52-40cd-b939-92f6ef167de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.346651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f155d98-0a52-40cd-b939-92f6ef167de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:50.348346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f155d98-0a52-40cd-b939-92f6ef167de9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:50.359109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1218/2409] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:50.363562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49983-a4d5-41ca-9924-45bddee0f364 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:50.374134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49983-a4d5-41ca-9924-45bddee0f364 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:50.380794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:50.381773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.382057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.382367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] PATH_INFO: `/v3/projects/4e462a27c1874ed1a589d0461b16e9e5/users/8bc4c5b40db0442fab1fdf9b8e44d087/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.399952 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e462a27c1874ed1a589d0461b16e9e5, user_id=8bc4c5b40db0442fab1fdf9b8e44d087, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:50.402660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:50.517925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1192/2410] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:50 2022] PUT /identity/v3/projects/4e462a27c1874ed1a589d0461b16e9e5/users/8bc4c5b40db0442fab1fdf9b8e44d087/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:50.528005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ac4ae-dc4b-4584-8daa-f1f99bd1c352 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.528288 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ac4ae-dc4b-4584-8daa-f1f99bd1c352 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.528560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ac4ae-dc4b-4584-8daa-f1f99bd1c352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.559331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-626ac4ae-dc4b-4584-8daa-f1f99bd1c352 None None] MFA Rules not processed for user `8bc4c5b40db0442fab1fdf9b8e44d087`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:50.560550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-626ac4ae-dc4b-4584-8daa-f1f99bd1c352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:50.593283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:50.598245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:50.668899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1219/2411] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:50.674078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ac4ae-dc4b-4584-8daa-f1f99bd1c352 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:50.710099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32eb28-2ee4-401b-a6ea-25c4c1e93550 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:50.717044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:50.717717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.717994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.718268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] PATH_INFO: `/v3/users/975a080756254318a4435e274d41bdba` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.719553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=975a080756254318a4435e274d41bdba)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:50.722062 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:50.725295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:50.749548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ac4ae-dc4b-4584-8daa-f1f99bd1c352 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:50.749689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:50.755010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:50.755836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.756193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.756528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.757226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:50.759351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:50.886520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1220/2412] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:50 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:50.892554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:50.894530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:50.969864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975a080756254318a4435e274d41bdba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.971233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975a080756254318a4435e274d41bdba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.979856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975a080756254318a4435e274d41bdba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.982276 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975a080756254318a4435e274d41bdba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:50.988531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cac23-95c9-48b2-9f4f-c44f0bd288ff admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:50.994863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:50.995619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:50.995887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:50.997075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] PATH_INFO: `/v3/projects/c04cc118470a457cb2f9d891f64a6d0d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:50.997706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04cc118470a457cb2f9d891f64a6d0d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:50.999705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:51.104664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975a080756254318a4435e274d41bdba'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:51.123264 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1193/2413] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:50 2022] DELETE /identity/v3/users/975a080756254318a4435e274d41bdba => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:51.133183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:51.133183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:51.146460 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04cc118470a457cb2f9d891f64a6d0d', 'request_id': 'req-e2df2316-af54-47b2-9d93-d5bde86c9918'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:51.213044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f1923-2984-4d5a-89b2-62b1d8f47a65 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:51.218240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:51.218849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:51.219162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:51.219437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:51.220625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:51.222569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:51.236529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04cc118470a457cb2f9d891f64a6d0d', 'request_id': 'req-e2df2316-af54-47b2-9d93-d5bde86c9918'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:51.246061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04cc118470a457cb2f9d891f64a6d0d', 'request_id': 'req-e2df2316-af54-47b2-9d93-d5bde86c9918'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:51.257336 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] The token cache is being invalidate because project c04cc118470a457cb2f9d891f64a6d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:51.257627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:51.258337 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2df2316-af54-47b2-9d93-d5bde86c9918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:51.259873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1221/2414] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:50 2022] DELETE /identity/v3/projects/c04cc118470a457cb2f9d891f64a6d0d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:51.271161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95614fb-ed99-469e-bb86-e2bea442e46c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:51.271161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95614fb-ed99-469e-bb86-e2bea442e46c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:51.271161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95614fb-ed99-469e-bb86-e2bea442e46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:51.284742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d95614fb-ed99-469e-bb86-e2bea442e46c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:51.303642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d95614fb-ed99-469e-bb86-e2bea442e46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:51.303642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1222/2415] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:51.306377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95614fb-ed99-469e-bb86-e2bea442e46c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:51.308141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d95614fb-ed99-469e-bb86-e2bea442e46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:51.362383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1194/2416] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:51 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:51.379822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:51.382488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:51.382709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:51.389227 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95614fb-ed99-469e-bb86-e2bea442e46c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:51.395077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc5a6-eea2-4212-8fb3-dd3e78d0372c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:51.398378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc5a6-eea2-4212-8fb3-dd3e78d0372c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:51.398378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc5a6-eea2-4212-8fb3-dd3e78d0372c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:51.398378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc5a6-eea2-4212-8fb3-dd3e78d0372c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:51.402855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d29b83-af00-45d6-85e7-af511044b4c4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:51.408518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc5a6-eea2-4212-8fb3-dd3e78d0372c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:51.410274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc5a6-eea2-4212-8fb3-dd3e78d0372c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:51.410825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713bba7-a0bc-4293-920e-2c0efc05cc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:51.411738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713bba7-a0bc-4293-920e-2c0efc05cc30 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:51.412125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713bba7-a0bc-4293-920e-2c0efc05cc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:51.412505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713bba7-a0bc-4293-920e-2c0efc05cc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:51.413271 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713bba7-a0bc-4293-920e-2c0efc05cc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:51.415691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713bba7-a0bc-4293-920e-2c0efc05cc30 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:51.422567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1223/2417] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:19:51.429868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1195/2418] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:51.436185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc5a6-eea2-4212-8fb3-dd3e78d0372c service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:51.453633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc5a6-eea2-4212-8fb3-dd3e78d0372c service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:51.461721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca0075-0481-4c99-bf84-e61fa78e925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:51.462397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca0075-0481-4c99-bf84-e61fa78e925c admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:51.462675 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca0075-0481-4c99-bf84-e61fa78e925c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:51.463086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca0075-0481-4c99-bf84-e61fa78e925c admin admin] PATH_INFO: `/v3/projects/3674db9a24a1499dbd0ef7dd32971d42/users/f707ae750e1f451ba9f44b8d1e74c732/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:51.483737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca0075-0481-4c99-bf84-e61fa78e925c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3674db9a24a1499dbd0ef7dd32971d42, user_id=f707ae750e1f451ba9f44b8d1e74c732, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:51.486522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca0075-0481-4c99-bf84-e61fa78e925c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:51.679057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1224/2419] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:51 2022] PUT /identity/v3/projects/3674db9a24a1499dbd0ef7dd32971d42/users/f707ae750e1f451ba9f44b8d1e74c732/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:51.682902 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713bba7-a0bc-4293-920e-2c0efc05cc30 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:51.693834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713bba7-a0bc-4293-920e-2c0efc05cc30 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:51.765094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d7247-30bf-465b-8cc5-355ab3044e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:51.766024 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d7247-30bf-465b-8cc5-355ab3044e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:51.766318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d7247-30bf-465b-8cc5-355ab3044e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:51.766591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d7247-30bf-465b-8cc5-355ab3044e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:51.770710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d7247-30bf-465b-8cc5-355ab3044e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:51.770710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d7247-30bf-465b-8cc5-355ab3044e48 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:51.781560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1196/2420] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:51.791565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca0075-0481-4c99-bf84-e61fa78e925c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:51.802970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca0075-0481-4c99-bf84-e61fa78e925c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:51.807807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:51.808453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:51.808728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:51.809001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] PATH_INFO: `/v3/projects/3674db9a24a1499dbd0ef7dd32971d42/users/f707ae750e1f451ba9f44b8d1e74c732/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:51.810904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3674db9a24a1499dbd0ef7dd32971d42, user_id=f707ae750e1f451ba9f44b8d1e74c732, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:51.813978 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:51.825199 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1225/2421] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:51 2022] PUT /identity/v3/projects/3674db9a24a1499dbd0ef7dd32971d42/users/f707ae750e1f451ba9f44b8d1e74c732/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:51.838317 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b39d-81de-479e-b0c4-0a926d7ac75b None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:51.838612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b39d-81de-479e-b0c4-0a926d7ac75b None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:51.847590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b39d-81de-479e-b0c4-0a926d7ac75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:51.882719 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:51.886457 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4389b39d-81de-479e-b0c4-0a926d7ac75b None None] MFA Rules not processed for user `f707ae750e1f451ba9f44b8d1e74c732`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:51.887786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4389b39d-81de-479e-b0c4-0a926d7ac75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:51.956321 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1197/2422] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:51.967138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:51.968826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:52.000539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389b39d-81de-479e-b0c4-0a926d7ac75b None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:52.002526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4389b39d-81de-479e-b0c4-0a926d7ac75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:52.046525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88653a24-1944-416a-a038-00c497fc9918 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:52.051027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a239f41-df07-425b-8277-f87e3a93ea12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:52.051599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a239f41-df07-425b-8277-f87e3a93ea12 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:52.051873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a239f41-df07-425b-8277-f87e3a93ea12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:52.052146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a239f41-df07-425b-8277-f87e3a93ea12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:52.054064 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6a239f41-df07-425b-8277-f87e3a93ea12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:52.093830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389b39d-81de-479e-b0c4-0a926d7ac75b None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:52.099433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:52.100435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:52.100783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:52.101128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] PATH_INFO: `/v3/projects/caf798435cb8497b9b70f941d04c3530` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:52.102385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf798435cb8497b9b70f941d04c3530)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:52.104613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:52.139825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a239f41-df07-425b-8277-f87e3a93ea12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:52.141411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a239f41-df07-425b-8277-f87e3a93ea12 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:52.151069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1226/2423] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:52.245998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf798435cb8497b9b70f941d04c3530', 'request_id': 'req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:52.251950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf798435cb8497b9b70f941d04c3530', 'request_id': 'req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:52.258572 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf798435cb8497b9b70f941d04c3530', 'request_id': 'req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:52.373354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] The token cache is being invalidate because project caf798435cb8497b9b70f941d04c3530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:52.373653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:52.380869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:52.380869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1198/2424] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:51 2022] DELETE /identity/v3/projects/caf798435cb8497b9b70f941d04c3530 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:19:53.705837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26023f9e-70da-4efb-8972-43bc122fbc85 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:53.706435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26023f9e-70da-4efb-8972-43bc122fbc85 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:53.706435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26023f9e-70da-4efb-8972-43bc122fbc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:53.725668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-26023f9e-70da-4efb-8972-43bc122fbc85 None None] MFA Rules not processed for user `b3c984088c0448f8abc0a0b60b820b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:53.726904 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-26023f9e-70da-4efb-8972-43bc122fbc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:53.787222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1227/2425] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:56.448566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:56.450054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:56.481197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:56.514534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8afe60-7e6b-492d-acce-4c2a3e7f6d87 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:56.519105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:56.519726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:56.520004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:56.520275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] PATH_INFO: `/v3/users/8bc4c5b40db0442fab1fdf9b8e44d087` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:56.521390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc4c5b40db0442fab1fdf9b8e44d087)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:56.523352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:56.783465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc4c5b40db0442fab1fdf9b8e44d087'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:56.784139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc4c5b40db0442fab1fdf9b8e44d087'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:56.791398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc4c5b40db0442fab1fdf9b8e44d087'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:56.792052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc4c5b40db0442fab1fdf9b8e44d087'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:56.886097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc4c5b40db0442fab1fdf9b8e44d087'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:56.905154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1199/2426] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:56 2022] DELETE /identity/v3/users/8bc4c5b40db0442fab1fdf9b8e44d087 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:56.910216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4202a41-1985-477b-ab6b-ee6f4de856d7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:56.910688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4202a41-1985-477b-ab6b-ee6f4de856d7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:56.910807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4202a41-1985-477b-ab6b-ee6f4de856d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:56.930977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f4202a41-1985-477b-ab6b-ee6f4de856d7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:56.932242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f4202a41-1985-477b-ab6b-ee6f4de856d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:56.937740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:56.997381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1228/2427] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:57.007465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:57.008824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:57.067503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc79b3c-9e25-490e-8c7d-c5eafe6412c7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:57.071116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a364f04-525f-4145-8826-84160dcb03ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:57.071745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a364f04-525f-4145-8826-84160dcb03ad service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:57.072025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a364f04-525f-4145-8826-84160dcb03ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:57.072296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a364f04-525f-4145-8826-84160dcb03ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:57.082010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a364f04-525f-4145-8826-84160dcb03ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:57.083484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a364f04-525f-4145-8826-84160dcb03ad service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:57.092984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1200/2428] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:19:57.525910 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4202a41-1985-477b-ab6b-ee6f4de856d7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:57.530082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f4202a41-1985-477b-ab6b-ee6f4de856d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:57.540734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4202a41-1985-477b-ab6b-ee6f4de856d7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:57.545625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:57.546276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:57.547150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:57.547605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] PATH_INFO: `/v3/projects/4e462a27c1874ed1a589d0461b16e9e5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:57.549272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e462a27c1874ed1a589d0461b16e9e5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:57.555077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:57.684207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e462a27c1874ed1a589d0461b16e9e5', 'request_id': 'req-efd6821b-73cf-4c90-babb-a5f6b6d1324b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:57.760126 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e462a27c1874ed1a589d0461b16e9e5', 'request_id': 'req-efd6821b-73cf-4c90-babb-a5f6b6d1324b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:57.766759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e462a27c1874ed1a589d0461b16e9e5', 'request_id': 'req-efd6821b-73cf-4c90-babb-a5f6b6d1324b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:57.781877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] The token cache is being invalidate because project 4e462a27c1874ed1a589d0461b16e9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:57.782317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:57.784403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:57.786236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1229/2429] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:57 2022] DELETE /identity/v3/projects/4e462a27c1874ed1a589d0461b16e9e5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:57.795789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a364f04-525f-4145-8826-84160dcb03ad service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:57.807529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a364f04-525f-4145-8826-84160dcb03ad service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:57.863297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:57.873142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:57.873929 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:57.874039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:57.874368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] PATH_INFO: `/v3/users/f707ae750e1f451ba9f44b8d1e74c732` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:57.875625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f707ae750e1f451ba9f44b8d1e74c732)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:57.877574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:58.046333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f707ae750e1f451ba9f44b8d1e74c732'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:58.047096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f707ae750e1f451ba9f44b8d1e74c732'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:58.054905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f707ae750e1f451ba9f44b8d1e74c732'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:58.057629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f707ae750e1f451ba9f44b8d1e74c732'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:58.163113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f2416feb-5776-4ee2-8a05-475d7952ab0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f707ae750e1f451ba9f44b8d1e74c732'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:58.187290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1201/2430] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:19:57 2022] DELETE /identity/v3/users/f707ae750e1f451ba9f44b8d1e74c732 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:58.741749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:58.743243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:58.761075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6821b-73cf-4c90-babb-a5f6b6d1324b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:58.844921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:58.845630 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:58.845841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:58.846134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] PATH_INFO: `/v3/projects/3674db9a24a1499dbd0ef7dd32971d42` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:58.847289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3674db9a24a1499dbd0ef7dd32971d42)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:58.849245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:58.948250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:58.957799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3674db9a24a1499dbd0ef7dd32971d42', 'request_id': 'req-b9d4809e-fc44-49c9-a270-27b0c58cc336'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:59.031146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3674db9a24a1499dbd0ef7dd32971d42', 'request_id': 'req-b9d4809e-fc44-49c9-a270-27b0c58cc336'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:59.037109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3674db9a24a1499dbd0ef7dd32971d42', 'request_id': 'req-b9d4809e-fc44-49c9-a270-27b0c58cc336'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:59.048290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] The token cache is being invalidate because project 3674db9a24a1499dbd0ef7dd32971d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:19:59.079346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:59.079346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b9d4809e-fc44-49c9-a270-27b0c58cc336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:19:59.079346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1230/2431] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:19:58 2022] DELETE /identity/v3/projects/3674db9a24a1499dbd0ef7dd32971d42 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:59.079677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216becbd-4e33-4c49-93be-e205b40be55f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.079677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216becbd-4e33-4c49-93be-e205b40be55f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.079677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216becbd-4e33-4c49-93be-e205b40be55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.079677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-216becbd-4e33-4c49-93be-e205b40be55f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:59.079677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-216becbd-4e33-4c49-93be-e205b40be55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:59.143100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1202/2432] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:59.147509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f851b06-5890-4b9b-a0aa-c8e7a90c0655 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.147876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f851b06-5890-4b9b-a0aa-c8e7a90c0655 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.148058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f851b06-5890-4b9b-a0aa-c8e7a90c0655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.168947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8f851b06-5890-4b9b-a0aa-c8e7a90c0655 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:59.170170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8f851b06-5890-4b9b-a0aa-c8e7a90c0655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:59.178104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1231/2433] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:59.182414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216becbd-4e33-4c49-93be-e205b40be55f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:59.192725 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216becbd-4e33-4c49-93be-e205b40be55f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:59.197536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:59.198156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.198431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.198702 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.199406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:19:59.199689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:59.201614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:59.214283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1203/2434] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:19:59.218847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f851b06-5890-4b9b-a0aa-c8e7a90c0655 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:59.228233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f851b06-5890-4b9b-a0aa-c8e7a90c0655 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:59.233247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788a3d3-a292-491b-abb1-48658333f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:59.233841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788a3d3-a292-491b-abb1-48658333f954 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.234123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788a3d3-a292-491b-abb1-48658333f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.234917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788a3d3-a292-491b-abb1-48658333f954 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.235599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788a3d3-a292-491b-abb1-48658333f954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:59.237502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788a3d3-a292-491b-abb1-48658333f954 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:59.389955 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1232/2435] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:19:59 2022] POST /identity/v3/projects => generated 363 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:19:59.394056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:59.405554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740aff4-de81-499d-9256-1c33812bddd8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:59.462008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:19:59.468338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65dc66-908e-4777-b55e-3594bb60f936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:59.468975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65dc66-908e-4777-b55e-3594bb60f936 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.469253 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65dc66-908e-4777-b55e-3594bb60f936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.469522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65dc66-908e-4777-b55e-3594bb60f936 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.470153 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65dc66-908e-4777-b55e-3594bb60f936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:59.472189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65dc66-908e-4777-b55e-3594bb60f936 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:59.591113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1204/2436] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:19:59 2022] POST /identity/v3/users => generated 282 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:19:59.596017 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788a3d3-a292-491b-abb1-48658333f954 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:59.606651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788a3d3-a292-491b-abb1-48658333f954 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:59.612626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81825f9c-9640-4415-9ca7-e3b5798698c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:59.613289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81825f9c-9640-4415-9ca7-e3b5798698c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.613567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81825f9c-9640-4415-9ca7-e3b5798698c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.613839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81825f9c-9640-4415-9ca7-e3b5798698c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.614529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81825f9c-9640-4415-9ca7-e3b5798698c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:59.616193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81825f9c-9640-4415-9ca7-e3b5798698c6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:59.626530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1233/2437] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:19:59.630686 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65dc66-908e-4777-b55e-3594bb60f936 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:59.642703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65dc66-908e-4777-b55e-3594bb60f936 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:59.646962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712865dc-8f07-4e91-abd3-8a8fdfe642a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:59.647675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712865dc-8f07-4e91-abd3-8a8fdfe642a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.647958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712865dc-8f07-4e91-abd3-8a8fdfe642a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.648233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712865dc-8f07-4e91-abd3-8a8fdfe642a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.648883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712865dc-8f07-4e91-abd3-8a8fdfe642a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:59.650659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712865dc-8f07-4e91-abd3-8a8fdfe642a2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:59.663323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1205/2438] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:19:59.667825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81825f9c-9640-4415-9ca7-e3b5798698c6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:59.678968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81825f9c-9640-4415-9ca7-e3b5798698c6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:19:59.683536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:19:59.684187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.684465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.684740 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] PATH_INFO: `/v3/projects/1e2f0692ef1949cca0ca278ccce0ba23/users/2dd6a4434b8b43b38a33d73bd9d797ad/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.707074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2f0692ef1949cca0ca278ccce0ba23, user_id=2dd6a4434b8b43b38a33d73bd9d797ad, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:19:59.707074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:19:59.861226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1234/2439] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:19:59 2022] PUT /identity/v3/projects/1e2f0692ef1949cca0ca278ccce0ba23/users/2dd6a4434b8b43b38a33d73bd9d797ad/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:19:59.862628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de67ba0-ca49-407f-ba7d-952730b1b460 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:19:59.862628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de67ba0-ca49-407f-ba7d-952730b1b460 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:19:59.862628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de67ba0-ca49-407f-ba7d-952730b1b460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:19:59.878786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4de67ba0-ca49-407f-ba7d-952730b1b460 None None] MFA Rules not processed for user `2dd6a4434b8b43b38a33d73bd9d797ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:19:59.880029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4de67ba0-ca49-407f-ba7d-952730b1b460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:19:59.954038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1206/2440] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:19:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:19:59.965693 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:19:59.967474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:00.032425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdc546-3b64-4462-b82e-b320c590324e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:00.037096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:00.037725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:00.038008 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:00.038283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:00.049030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:00.050491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:00.064796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1235/2441] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:19:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:20:09.949879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:10.062056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:16.899678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b029a2-cb59-4f19-8f43-24bfdaef85fb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:16.900096 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b029a2-cb59-4f19-8f43-24bfdaef85fb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:16.900196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b029a2-cb59-4f19-8f43-24bfdaef85fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:16.921604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-36b029a2-cb59-4f19-8f43-24bfdaef85fb None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:16.922817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-36b029a2-cb59-4f19-8f43-24bfdaef85fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:16.980460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1207/2442] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:20:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:20:17.021840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:17.023276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:17.096782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511ef20-6385-4406-afee-d60fc9f85c0c service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:17.102151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb5565-01eb-4a46-b745-63c5f814cb0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:17.102755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb5565-01eb-4a46-b745-63c5f814cb0c service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:17.103073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb5565-01eb-4a46-b745-63c5f814cb0c service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:17.103349 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb5565-01eb-4a46-b745-63c5f814cb0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:17.112029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb5565-01eb-4a46-b745-63c5f814cb0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:17.113617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb5565-01eb-4a46-b745-63c5f814cb0c service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:17.122072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1236/2443] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:20:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:20:26.973575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=25 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:27.118970 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=32 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:42.288537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bcc39-829c-4f80-a55f-c0ce3c56d099 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:42.288992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bcc39-829c-4f80-a55f-c0ce3c56d099 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:42.289173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bcc39-829c-4f80-a55f-c0ce3c56d099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:42.311018 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-534bcc39-829c-4f80-a55f-c0ce3c56d099 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:42.312295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-534bcc39-829c-4f80-a55f-c0ce3c56d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:42.320232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1208/2444] 10.222.0.58 () {58 vars in 967 bytes} [Tue Aug 23 08:20:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:20:42.334156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb5565-01eb-4a46-b745-63c5f814cb0c service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:42.345471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb5565-01eb-4a46-b745-63c5f814cb0c service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:42.350673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f12c92-8ec9-421a-9082-7758ed5e32cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:42.351302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f12c92-8ec9-421a-9082-7758ed5e32cd service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:42.355425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f12c92-8ec9-421a-9082-7758ed5e32cd service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:42.355749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f12c92-8ec9-421a-9082-7758ed5e32cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:42.364427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f12c92-8ec9-421a-9082-7758ed5e32cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:42.366071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f12c92-8ec9-421a-9082-7758ed5e32cd service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:42.375320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1237/2445] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:20:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:20:52.304856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:52.371809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:54.213165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bcc39-829c-4f80-a55f-c0ce3c56d099 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:54.215480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-534bcc39-829c-4f80-a55f-c0ce3c56d099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:54.284509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bcc39-829c-4f80-a55f-c0ce3c56d099 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:54.289126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:54.296782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:54.296782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:54.296782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] PATH_INFO: `/v3/users/45777028be184247b0a58eee7d2c7e45` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:54.296782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45777028be184247b0a58eee7d2c7e45)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:54.296782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:54.525014 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45777028be184247b0a58eee7d2c7e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:54.525671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45777028be184247b0a58eee7d2c7e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:54.532785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45777028be184247b0a58eee7d2c7e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:54.533422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45777028be184247b0a58eee7d2c7e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:54.616855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-4e69e2b3-58e2-4357-b937-53efed318f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45777028be184247b0a58eee7d2c7e45'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:54.636820 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1209/2446] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:20:54 2022] DELETE /identity/v3/users/45777028be184247b0a58eee7d2c7e45 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:20:54.911859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f12c92-8ec9-421a-9082-7758ed5e32cd service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:54.928236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f12c92-8ec9-421a-9082-7758ed5e32cd service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:54.993861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:54.994478 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:54.994753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:54.995052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] PATH_INFO: `/v3/projects/cdbd04906d2b42fba52e61f3dbd0af21` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:54.996159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdbd04906d2b42fba52e61f3dbd0af21)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:54.998058 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:55.077847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdbd04906d2b42fba52e61f3dbd0af21', 'request_id': 'req-23376db7-b871-46d9-8850-73890c56c7d4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.145559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdbd04906d2b42fba52e61f3dbd0af21', 'request_id': 'req-23376db7-b871-46d9-8850-73890c56c7d4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.152245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdbd04906d2b42fba52e61f3dbd0af21', 'request_id': 'req-23376db7-b871-46d9-8850-73890c56c7d4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.163714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] The token cache is being invalidate because project cdbd04906d2b42fba52e61f3dbd0af21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:20:55.168851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.168851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23376db7-b871-46d9-8850-73890c56c7d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.168851 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1238/2447] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:20:54 2022] DELETE /identity/v3/projects/cdbd04906d2b42fba52e61f3dbd0af21 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:20:55.175482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b959e-ed9b-4da8-89e8-7a269d51503d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.175762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b959e-ed9b-4da8-89e8-7a269d51503d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.176029 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b959e-ed9b-4da8-89e8-7a269d51503d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.195769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e94b959e-ed9b-4da8-89e8-7a269d51503d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:55.197382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e94b959e-ed9b-4da8-89e8-7a269d51503d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:55.256444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1210/2448] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:20:55.265174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a332-7535-43bf-94d7-6695dfd9b1e4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.265457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a332-7535-43bf-94d7-6695dfd9b1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.265732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a332-7535-43bf-94d7-6695dfd9b1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.285877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3d79a332-7535-43bf-94d7-6695dfd9b1e4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:55.287095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3d79a332-7535-43bf-94d7-6695dfd9b1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:55.295217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1239/2449] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:20:55.299646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b959e-ed9b-4da8-89e8-7a269d51503d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:55.311002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b959e-ed9b-4da8-89e8-7a269d51503d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:55.315891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:55.316503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.316813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.317089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.317785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:20:55.318090 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:55.320001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:55.332948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1211/2450] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 08:20:55.338390 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79a332-7535-43bf-94d7-6695dfd9b1e4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:55.347821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79a332-7535-43bf-94d7-6695dfd9b1e4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:55.352449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:55.353085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.353360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.353631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.354289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:55.356212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:55.469096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1240/2451] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:20:55 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:20:55.480644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:55.492152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b129f6-32de-4e84-9061-6fd255e1180b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:55.495346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:55.497218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:55.498824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:55.552891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f086b85-4b47-416b-9bac-a823af01a6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:55.553496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f086b85-4b47-416b-9bac-a823af01a6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.553777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f086b85-4b47-416b-9bac-a823af01a6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.554052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f086b85-4b47-416b-9bac-a823af01a6c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.554677 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f086b85-4b47-416b-9bac-a823af01a6c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:55.556632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f086b85-4b47-416b-9bac-a823af01a6c5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:55.563934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdecf4-81c7-4495-9a36-edb5c13379e5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:55.569176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:55.569799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.570078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.570357 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] PATH_INFO: `/v3/users/515ff673dc394aceae078cb52ca607dd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.571553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=515ff673dc394aceae078cb52ca607dd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:55.574175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:55.576586 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:55.672094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1212/2452] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:20:55 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:20:55.679501 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f086b85-4b47-416b-9bac-a823af01a6c5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:55.689606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f086b85-4b47-416b-9bac-a823af01a6c5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:55.694723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb527c-fd9c-4547-a330-f386da341c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:55.695327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb527c-fd9c-4547-a330-f386da341c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.700128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb527c-fd9c-4547-a330-f386da341c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.700128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb527c-fd9c-4547-a330-f386da341c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.700128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb527c-fd9c-4547-a330-f386da341c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:55.700128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb527c-fd9c-4547-a330-f386da341c5f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:55.709324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1213/2453] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:20:55.713075 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb527c-fd9c-4547-a330-f386da341c5f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:55.722469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb527c-fd9c-4547-a330-f386da341c5f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:55.727202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec6b15-e8e4-4641-baf1-e753dcc180d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:55.727767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6b15-e8e4-4641-baf1-e753dcc180d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.728044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6b15-e8e4-4641-baf1-e753dcc180d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.728316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6b15-e8e4-4641-baf1-e753dcc180d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.729421 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec6b15-e8e4-4641-baf1-e753dcc180d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:55.730984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec6b15-e8e4-4641-baf1-e753dcc180d1 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:55.740628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1214/2454] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:20:55.745379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec6b15-e8e4-4641-baf1-e753dcc180d1 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:55.754905 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec6b15-e8e4-4641-baf1-e753dcc180d1 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:55.759883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa533d-5363-48cf-bd9e-915386701f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:55.763352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa533d-5363-48cf-bd9e-915386701f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.763352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa533d-5363-48cf-bd9e-915386701f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.763352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa533d-5363-48cf-bd9e-915386701f67 admin admin] PATH_INFO: `/v3/projects/ee175b1f398844f6be0075d3e127e8b3/users/a0a59ef94e054c3d8175e1c0f1ad783a/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.775449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa533d-5363-48cf-bd9e-915386701f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee175b1f398844f6be0075d3e127e8b3, user_id=a0a59ef94e054c3d8175e1c0f1ad783a, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:55.778155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa533d-5363-48cf-bd9e-915386701f67 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:55.825359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '515ff673dc394aceae078cb52ca607dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.832444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '515ff673dc394aceae078cb52ca607dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.833110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '515ff673dc394aceae078cb52ca607dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.833722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '515ff673dc394aceae078cb52ca607dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.845766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-88d67631-fe4a-40d4-ba55-d32e5bade349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '515ff673dc394aceae078cb52ca607dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:55.880142 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1215/2455] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:20:55 2022] PUT /identity/v3/projects/ee175b1f398844f6be0075d3e127e8b3/users/a0a59ef94e054c3d8175e1c0f1ad783a/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:20:55.884983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2eb4-f0d2-4b42-8294-e079025b1d5a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.885272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2eb4-f0d2-4b42-8294-e079025b1d5a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.885546 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2eb4-f0d2-4b42-8294-e079025b1d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.904081 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ec5c2eb4-f0d2-4b42-8294-e079025b1d5a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:55.906858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ec5c2eb4-f0d2-4b42-8294-e079025b1d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:55.960326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1241/2456] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:20:55 2022] DELETE /identity/v3/users/515ff673dc394aceae078cb52ca607dd => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:20:55.964929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b3ed-ad18-4ef0-af75-d56b42cb9871 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:55.965198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b3ed-ad18-4ef0-af75-d56b42cb9871 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:55.965471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750b3ed-ad18-4ef0-af75-d56b42cb9871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:55.968441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1216/2457] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:20:55.979586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c2eb4-f0d2-4b42-8294-e079025b1d5a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:55.980859 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ec5c2eb4-f0d2-4b42-8294-e079025b1d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:55.984785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1750b3ed-ad18-4ef0-af75-d56b42cb9871 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:55.985965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1750b3ed-ad18-4ef0-af75-d56b42cb9871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:56.038643 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c2eb4-f0d2-4b42-8294-e079025b1d5a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:56.042922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:56.045096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1242/2458] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:20:56.055385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750b3ed-ad18-4ef0-af75-d56b42cb9871 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:56.089756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:56.090442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:56.090722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:56.091032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:56.100716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:56.102279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:56.120951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1217/2459] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:20:56.123549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750b3ed-ad18-4ef0-af75-d56b42cb9871 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:56.127394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd5e34-5a1e-4b28-9ad3-9e3391fd7a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:56.128003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd5e34-5a1e-4b28-9ad3-9e3391fd7a78 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:56.128311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd5e34-5a1e-4b28-9ad3-9e3391fd7a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:56.128615 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd5e34-5a1e-4b28-9ad3-9e3391fd7a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:56.138999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd5e34-5a1e-4b28-9ad3-9e3391fd7a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:56.140536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd5e34-5a1e-4b28-9ad3-9e3391fd7a78 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:56.150148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1243/2460] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:20:56.671232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:56.674108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:56.691331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e2446-a795-4d7c-893d-c11bd0c6c13b service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:56.697020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:56.697814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:56.698185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:56.698694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] PATH_INFO: `/v3/projects/333a97c605cc42ea8865885bfdaecee1` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:56.699943 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] RBAC: Authorizing `identity:delete_project(project_id=333a97c605cc42ea8865885bfdaecee1)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:56.701930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:56.841474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333a97c605cc42ea8865885bfdaecee1', 'request_id': 'req-25a11218-67cf-47c1-b285-8e5eaf9ea5af'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:56.847118 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333a97c605cc42ea8865885bfdaecee1', 'request_id': 'req-25a11218-67cf-47c1-b285-8e5eaf9ea5af'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:56.854174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333a97c605cc42ea8865885bfdaecee1', 'request_id': 'req-25a11218-67cf-47c1-b285-8e5eaf9ea5af'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:57.008116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] The token cache is being invalidate because project 333a97c605cc42ea8865885bfdaecee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:20:57.012323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:57.012323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-25a11218-67cf-47c1-b285-8e5eaf9ea5af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:20:57.012323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1218/2461] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:20:56 2022] DELETE /identity/v3/projects/333a97c605cc42ea8865885bfdaecee1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:20:57.015374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30bf62-76b8-45be-adea-5ea5e96134d5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.015828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30bf62-76b8-45be-adea-5ea5e96134d5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.015925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30bf62-76b8-45be-adea-5ea5e96134d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.039412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dc30bf62-76b8-45be-adea-5ea5e96134d5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:57.042603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dc30bf62-76b8-45be-adea-5ea5e96134d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:57.064766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:57.110175 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1244/2462] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:20:57.118011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc75ae2-e065-419d-b052-bf17d0be88e0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.118299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc75ae2-e065-419d-b052-bf17d0be88e0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.118572 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc75ae2-e065-419d-b052-bf17d0be88e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.138517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0cc75ae2-e065-419d-b052-bf17d0be88e0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:57.139817 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0cc75ae2-e065-419d-b052-bf17d0be88e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:57.147866 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1219/2463] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:20:57.152573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30bf62-76b8-45be-adea-5ea5e96134d5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:57.162376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30bf62-76b8-45be-adea-5ea5e96134d5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:57.167300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:57.167900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.168178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.168443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.169081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:20:57.169380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:57.171122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:57.187524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1245/2464] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:20:57.189647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc75ae2-e065-419d-b052-bf17d0be88e0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:57.200104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc75ae2-e065-419d-b052-bf17d0be88e0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:57.204841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeeacc0-1a6a-4a81-bcf9-9edd41217de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:57.205437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeacc0-1a6a-4a81-bcf9-9edd41217de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.205739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeacc0-1a6a-4a81-bcf9-9edd41217de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.206037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeacc0-1a6a-4a81-bcf9-9edd41217de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.206678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeeacc0-1a6a-4a81-bcf9-9edd41217de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:57.208773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeeacc0-1a6a-4a81-bcf9-9edd41217de4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:57.317384 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1220/2465] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:20:57 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:20:57.320200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:57.330136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea527-3100-4e60-98fc-ab7a04d5f57b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:57.389290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e3fc5-037d-41f9-8c7c-81007c1efbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:57.389915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e3fc5-037d-41f9-8c7c-81007c1efbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.390186 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e3fc5-037d-41f9-8c7c-81007c1efbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.390459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e3fc5-037d-41f9-8c7c-81007c1efbc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.391146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e3fc5-037d-41f9-8c7c-81007c1efbc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:57.393291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e3fc5-037d-41f9-8c7c-81007c1efbc9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:57.528984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1246/2466] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:20:57 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:20:57.530557 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeeacc0-1a6a-4a81-bcf9-9edd41217de4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:57.541619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeeacc0-1a6a-4a81-bcf9-9edd41217de4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:57.546425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194da580-3d02-44ff-94e9-e9710f85a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:57.547080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194da580-3d02-44ff-94e9-e9710f85a3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.547360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194da580-3d02-44ff-94e9-e9710f85a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.547630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194da580-3d02-44ff-94e9-e9710f85a3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.548266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194da580-3d02-44ff-94e9-e9710f85a3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:57.549898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194da580-3d02-44ff-94e9-e9710f85a3bd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:57.561139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1221/2467] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:20:57.565269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e3fc5-037d-41f9-8c7c-81007c1efbc9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:57.585501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e3fc5-037d-41f9-8c7c-81007c1efbc9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:57.592140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392869b1-5749-4cde-8a67-b0bdf8a2fe08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:57.592746 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392869b1-5749-4cde-8a67-b0bdf8a2fe08 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.593020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392869b1-5749-4cde-8a67-b0bdf8a2fe08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.593287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392869b1-5749-4cde-8a67-b0bdf8a2fe08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.593925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392869b1-5749-4cde-8a67-b0bdf8a2fe08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:57.595745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392869b1-5749-4cde-8a67-b0bdf8a2fe08 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:57.607510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1247/2468] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:20:57.611689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194da580-3d02-44ff-94e9-e9710f85a3bd admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:57.621772 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194da580-3d02-44ff-94e9-e9710f85a3bd admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:57.626329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9495af9-b2ee-4916-a58b-10bc625385fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:57.627003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495af9-b2ee-4916-a58b-10bc625385fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.630843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495af9-b2ee-4916-a58b-10bc625385fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.630843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495af9-b2ee-4916-a58b-10bc625385fe admin admin] PATH_INFO: `/v3/projects/1e8586b18aaa47f598a11b272031217a/users/fe286cbe88934741822be3f74870b302/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.642327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9495af9-b2ee-4916-a58b-10bc625385fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8586b18aaa47f598a11b272031217a, user_id=fe286cbe88934741822be3f74870b302, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:57.645191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9495af9-b2ee-4916-a58b-10bc625385fe admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:57.659182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb439f-7fd5-4f58-92d4-e8940d57b9f4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.659473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb439f-7fd5-4f58-92d4-e8940d57b9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.659929 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb439f-7fd5-4f58-92d4-e8940d57b9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.695141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a0bb439f-7fd5-4f58-92d4-e8940d57b9f4 None None] MFA Rules not processed for user `a0a59ef94e054c3d8175e1c0f1ad783a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:57.695438 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a0bb439f-7fd5-4f58-92d4-e8940d57b9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:57.721612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1222/2469] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:20:57 2022] PUT /identity/v3/projects/1e8586b18aaa47f598a11b272031217a/users/fe286cbe88934741822be3f74870b302/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:20:57.732414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972df07-0bb9-4c5f-8c51-1a7c7ec79559 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.732731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972df07-0bb9-4c5f-8c51-1a7c7ec79559 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.733199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972df07-0bb9-4c5f-8c51-1a7c7ec79559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.766757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b972df07-0bb9-4c5f-8c51-1a7c7ec79559 None None] MFA Rules not processed for user `fe286cbe88934741822be3f74870b302`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:20:57.768083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b972df07-0bb9-4c5f-8c51-1a7c7ec79559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:57.789882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1248/2470] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:20:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:20:57.792437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:57.804850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb439f-7fd5-4f58-92d4-e8940d57b9f4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:57.806480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a0bb439f-7fd5-4f58-92d4-e8940d57b9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:57.839730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:20:57.846801 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1223/2471] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:20:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:20:57.858655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972df07-0bb9-4c5f-8c51-1a7c7ec79559 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:20:57.860161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b972df07-0bb9-4c5f-8c51-1a7c7ec79559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:20:57.882889 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb439f-7fd5-4f58-92d4-e8940d57b9f4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:57.929632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972df07-0bb9-4c5f-8c51-1a7c7ec79559 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:20:57.933515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95a31a-d62d-4bed-b3ac-6495bc55003a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:57.934058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95a31a-d62d-4bed-b3ac-6495bc55003a service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.934343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95a31a-d62d-4bed-b3ac-6495bc55003a service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.934613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95a31a-d62d-4bed-b3ac-6495bc55003a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.941428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf144c-6223-4628-9a68-b6815710c52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:20:57.942102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf144c-6223-4628-9a68-b6815710c52d service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:20:57.942379 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf144c-6223-4628-9a68-b6815710c52d service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:20:57.942642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf144c-6223-4628-9a68-b6815710c52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:20:57.944360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95a31a-d62d-4bed-b3ac-6495bc55003a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:57.945818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95a31a-d62d-4bed-b3ac-6495bc55003a service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:57.956575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1224/2472] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:20:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:20:58.000384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf144c-6223-4628-9a68-b6815710c52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:20:58.002004 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf144c-6223-4628-9a68-b6815710c52d service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:20:58.015185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1249/2473] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:20:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:21:05.000437 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7863b4-63d1-475d-9bdf-63b91c14c5e5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:05.000925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7863b4-63d1-475d-9bdf-63b91c14c5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:05.000925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7863b4-63d1-475d-9bdf-63b91c14c5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:05.020008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-db7863b4-63d1-475d-9bdf-63b91c14c5e5 None None] MFA Rules not processed for user `a0a59ef94e054c3d8175e1c0f1ad783a`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:05.025108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-db7863b4-63d1-475d-9bdf-63b91c14c5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:05.028104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1225/2474] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:05.086307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc273-a9ac-4777-a5b3-1caf3abfc711 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:05.086307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc273-a9ac-4777-a5b3-1caf3abfc711 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:05.086307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc273-a9ac-4777-a5b3-1caf3abfc711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:05.101484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8c4fc273-a9ac-4777-a5b3-1caf3abfc711 None None] MFA Rules not processed for user `fe286cbe88934741822be3f74870b302`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:05.103665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8c4fc273-a9ac-4777-a5b3-1caf3abfc711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:05.110988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1250/2475] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:06.773864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7863b4-63d1-475d-9bdf-63b91c14c5e5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:06.776467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-db7863b4-63d1-475d-9bdf-63b91c14c5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:06.856516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7863b4-63d1-475d-9bdf-63b91c14c5e5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:06.862147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b98b3-de80-408b-a8a4-2a5de7651a16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:06.862776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b98b3-de80-408b-a8a4-2a5de7651a16 service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:06.863085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b98b3-de80-408b-a8a4-2a5de7651a16 service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:06.863354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b98b3-de80-408b-a8a4-2a5de7651a16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:06.865101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b34b98b3-de80-408b-a8a4-2a5de7651a16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:06.877212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b98b3-de80-408b-a8a4-2a5de7651a16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:06.878780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b98b3-de80-408b-a8a4-2a5de7651a16 service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:06.894251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1226/2476] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:21:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:21:11.606085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fc273-a9ac-4777-a5b3-1caf3abfc711 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:11.609723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8c4fc273-a9ac-4777-a5b3-1caf3abfc711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:11.686619 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fc273-a9ac-4777-a5b3-1caf3abfc711 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:11.691676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:11.692809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:11.693159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:11.693521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] PATH_INFO: `/v3/users/adfe865874114e618d1f208ffa685762` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:11.694772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=adfe865874114e618d1f208ffa685762)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:11.697219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:11.865720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:11.956044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adfe865874114e618d1f208ffa685762'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:11.963739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adfe865874114e618d1f208ffa685762'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:11.964585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adfe865874114e618d1f208ffa685762'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:11.965155 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adfe865874114e618d1f208ffa685762'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:11.989688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adfe865874114e618d1f208ffa685762'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:12.073618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1251/2477] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:21:11 2022] DELETE /identity/v3/users/adfe865874114e618d1f208ffa685762 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:21:12.078123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b17d-2078-4ff9-824e-c39bc0854364 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:12.078598 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b17d-2078-4ff9-824e-c39bc0854364 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:12.080781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b17d-2078-4ff9-824e-c39bc0854364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:12.098310 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c2f6b17d-2078-4ff9-824e-c39bc0854364 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:12.100162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c2f6b17d-2078-4ff9-824e-c39bc0854364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:12.148403 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:12.171263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1227/2478] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:12.184461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:12.189636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:12.254662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd6854-7dc9-4604-8ce8-35966d3a74fe admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:12.259668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208da4b-3253-4d53-a092-84628e0cc8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:12.260325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208da4b-3253-4d53-a092-84628e0cc8ec service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:12.260605 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208da4b-3253-4d53-a092-84628e0cc8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:12.260879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208da4b-3253-4d53-a092-84628e0cc8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:12.270710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208da4b-3253-4d53-a092-84628e0cc8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:12.272163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208da4b-3253-4d53-a092-84628e0cc8ec service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:12.282719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1252/2479] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:21:13.118394 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6b17d-2078-4ff9-824e-c39bc0854364 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:13.121921 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c2f6b17d-2078-4ff9-824e-c39bc0854364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:13.137213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6b17d-2078-4ff9-824e-c39bc0854364 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:13.142354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:13.143185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:13.143461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:13.143730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] PATH_INFO: `/v3/projects/0db1595fa58d47f7ab3ae158b29a0e49` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:13.144980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db1595fa58d47f7ab3ae158b29a0e49)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:13.147482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:13.286267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db1595fa58d47f7ab3ae158b29a0e49', 'request_id': 'req-5a046ca5-2426-42e6-9ae9-099a08d8aa70'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:13.292474 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db1595fa58d47f7ab3ae158b29a0e49', 'request_id': 'req-5a046ca5-2426-42e6-9ae9-099a08d8aa70'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:13.299479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db1595fa58d47f7ab3ae158b29a0e49', 'request_id': 'req-5a046ca5-2426-42e6-9ae9-099a08d8aa70'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:13.396088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] The token cache is being invalidate because project 0db1595fa58d47f7ab3ae158b29a0e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:21:13.396088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:13.396088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:13.396088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1228/2480] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:21:13 2022] DELETE /identity/v3/projects/0db1595fa58d47f7ab3ae158b29a0e49 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:21:22.544593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=6,keystone:SELECT=45,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:23.383063 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:33.065145 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93af9-9aed-40e4-b96e-88376e2ab338 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:33.065623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93af9-9aed-40e4-b96e-88376e2ab338 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:33.065736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93af9-9aed-40e4-b96e-88376e2ab338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:33.086392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b1c93af9-9aed-40e4-b96e-88376e2ab338 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:33.087633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b1c93af9-9aed-40e4-b96e-88376e2ab338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:33.151404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1253/2481] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:21:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:33.177442 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:33.179266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:33.250890 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a046ca5-2426-42e6-9ae9-099a08d8aa70 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:33.255971 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbb5da-cd8c-4b06-ad15-d62e86457efd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:33.256574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbb5da-cd8c-4b06-ad15-d62e86457efd service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:33.256854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbb5da-cd8c-4b06-ad15-d62e86457efd service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:33.257126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbb5da-cd8c-4b06-ad15-d62e86457efd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:33.265505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbb5da-cd8c-4b06-ad15-d62e86457efd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:33.267218 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbb5da-cd8c-4b06-ad15-d62e86457efd service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:33.275509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1229/2482] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:21:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:21:34.187781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c93af9-9aed-40e4-b96e-88376e2ab338 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:34.189828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b1c93af9-9aed-40e4-b96e-88376e2ab338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:34.259138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c93af9-9aed-40e4-b96e-88376e2ab338 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:34.263663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:34.264293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:34.264572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:34.264844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] PATH_INFO: `/v3/users/2dd6a4434b8b43b38a33d73bd9d797ad` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:34.265984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd6a4434b8b43b38a33d73bd9d797ad)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:34.267965 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:34.313842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752b82-fa90-4605-a52b-acee327df9a0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:34.314291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752b82-fa90-4605-a52b-acee327df9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:34.314413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05752b82-fa90-4605-a52b-acee327df9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:34.335747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-05752b82-fa90-4605-a52b-acee327df9a0 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:34.337000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-05752b82-fa90-4605-a52b-acee327df9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:34.344372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1230/2483] 10.222.0.58 () {58 vars in 967 bytes} [Tue Aug 23 08:21:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:21:34.361076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05752b82-fa90-4605-a52b-acee327df9a0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:34.370355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05752b82-fa90-4605-a52b-acee327df9a0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:34.375672 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f3bd3-1134-43db-8955-dc31e200f531 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:34.376290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3bd3-1134-43db-8955-dc31e200f531 service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:34.376585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3bd3-1134-43db-8955-dc31e200f531 service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:34.376842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3bd3-1134-43db-8955-dc31e200f531 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:34.385865 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f3bd3-1134-43db-8955-dc31e200f531 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:34.387703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f3bd3-1134-43db-8955-dc31e200f531 service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:34.396735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1231/2484] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:21:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 08:21:34.442614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd6a4434b8b43b38a33d73bd9d797ad'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:34.450563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd6a4434b8b43b38a33d73bd9d797ad'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:34.451358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd6a4434b8b43b38a33d73bd9d797ad'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:34.451998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd6a4434b8b43b38a33d73bd9d797ad'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:34.463852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd6a4434b8b43b38a33d73bd9d797ad'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:34.519594 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1254/2485] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:21:34 2022] DELETE /identity/v3/users/2dd6a4434b8b43b38a33d73bd9d797ad => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:21:34.524072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a842-ccaf-48ae-bb04-c2b161de7a7c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:34.524361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a842-ccaf-48ae-bb04-c2b161de7a7c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:34.524640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a842-ccaf-48ae-bb04-c2b161de7a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:34.545164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-bec8a842-ccaf-48ae-bb04-c2b161de7a7c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:34.546386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-bec8a842-ccaf-48ae-bb04-c2b161de7a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:34.606899 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1232/2486] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:34.616743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:34.618711 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:34.648369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:34.679502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17086bb6-df56-4b3d-87af-66191f9da585 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:34.683525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:34.684239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:34.684484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:34.684813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:34.698077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:34.699648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:34.709164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1255/2487] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:21:35.104648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abed221-cc09-4efb-b370-480402225c7c None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.106875 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abed221-cc09-4efb-b370-480402225c7c None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.107538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abed221-cc09-4efb-b370-480402225c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.139082 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9abed221-cc09-4efb-b370-480402225c7c None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:35.140389 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9abed221-cc09-4efb-b370-480402225c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:35.198468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1233/2488] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:21:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:35.210271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:35.212293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:35.222316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abed221-cc09-4efb-b370-480402225c7c None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:35.225106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9abed221-cc09-4efb-b370-480402225c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:35.245347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abed221-cc09-4efb-b370-480402225c7c None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:35.250885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:35.254181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.254181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.254181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] PATH_INFO: `/v3/projects/1e2f0692ef1949cca0ca278ccce0ba23` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.254181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2f0692ef1949cca0ca278ccce0ba23)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:35.255456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:35.269201 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:35.292010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172144a-6de9-4c6c-96bd-bfe678a89c04 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:35.297584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab1af5-a473-4540-a690-ee8bd4499e94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:35.298197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab1af5-a473-4540-a690-ee8bd4499e94 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.298473 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab1af5-a473-4540-a690-ee8bd4499e94 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.298745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab1af5-a473-4540-a690-ee8bd4499e94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.307502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab1af5-a473-4540-a690-ee8bd4499e94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:35.309166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab1af5-a473-4540-a690-ee8bd4499e94 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:35.317612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1256/2489] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:21:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:21:35.379838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2f0692ef1949cca0ca278ccce0ba23', 'request_id': 'req-f18dd765-26bb-43f3-8243-380e50dcff7f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:35.386126 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2f0692ef1949cca0ca278ccce0ba23', 'request_id': 'req-f18dd765-26bb-43f3-8243-380e50dcff7f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:35.392944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2f0692ef1949cca0ca278ccce0ba23', 'request_id': 'req-f18dd765-26bb-43f3-8243-380e50dcff7f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:35.465287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] The token cache is being invalidate because project 1e2f0692ef1949cca0ca278ccce0ba23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:21:35.465602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:35.466263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f18dd765-26bb-43f3-8243-380e50dcff7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:35.467892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1234/2490] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:21:35 2022] DELETE /identity/v3/projects/1e2f0692ef1949cca0ca278ccce0ba23 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:21:35.472868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf4918-492a-42c3-8790-555116f90a2c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.473154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf4918-492a-42c3-8790-555116f90a2c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.473464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf4918-492a-42c3-8790-555116f90a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.494053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6eaf4918-492a-42c3-8790-555116f90a2c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:35.495425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6eaf4918-492a-42c3-8790-555116f90a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:35.557022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1257/2491] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:35.565141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efb8a5-83bf-46e1-9795-b24a66e0a01d None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.565433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efb8a5-83bf-46e1-9795-b24a66e0a01d None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.565749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efb8a5-83bf-46e1-9795-b24a66e0a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.587960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a0efb8a5-83bf-46e1-9795-b24a66e0a01d None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:35.589204 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a0efb8a5-83bf-46e1-9795-b24a66e0a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:35.597261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1235/2492] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:35.608566 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf4918-492a-42c3-8790-555116f90a2c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:35.618599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf4918-492a-42c3-8790-555116f90a2c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:35.623177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:35.623772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.624051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.624322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.625004 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:21:35.625308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:35.626910 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:35.639681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1258/2493] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:21:35.644200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efb8a5-83bf-46e1-9795-b24a66e0a01d None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:35.654347 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efb8a5-83bf-46e1-9795-b24a66e0a01d None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:35.659161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e4a1e-2fd0-45d6-8d99-cf66763fa6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:35.659754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e4a1e-2fd0-45d6-8d99-cf66763fa6ae admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.660031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e4a1e-2fd0-45d6-8d99-cf66763fa6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.666827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e4a1e-2fd0-45d6-8d99-cf66763fa6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.666827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e4a1e-2fd0-45d6-8d99-cf66763fa6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:35.666827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e4a1e-2fd0-45d6-8d99-cf66763fa6ae admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:35.789676 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1236/2494] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:21:35 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:21:35.792762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:35.803692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999a073-c498-4753-96c5-ead24d08b5cf admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:35.852458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:35.874130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad32dc8-060d-496b-b07d-dd8b4423b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:35.874868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad32dc8-060d-496b-b07d-dd8b4423b621 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.875277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad32dc8-060d-496b-b07d-dd8b4423b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.875646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad32dc8-060d-496b-b07d-dd8b4423b621 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.876372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad32dc8-060d-496b-b07d-dd8b4423b621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:35.878448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad32dc8-060d-496b-b07d-dd8b4423b621 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:35.957373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9968f9-b2c4-4204-9c0b-a25670186f10 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:35.957653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9968f9-b2c4-4204-9c0b-a25670186f10 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:35.958912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9968f9-b2c4-4204-9c0b-a25670186f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:35.977330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6d9968f9-b2c4-4204-9c0b-a25670186f10 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:35.978585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6d9968f9-b2c4-4204-9c0b-a25670186f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:35.988248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1259/2495] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:21:35 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:21:35.991746 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad32dc8-060d-496b-b07d-dd8b4423b621 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:36.003046 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad32dc8-060d-496b-b07d-dd8b4423b621 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:36.008386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc9150-c444-428f-a456-5d01e1bd67a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:36.008996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc9150-c444-428f-a456-5d01e1bd67a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:36.009280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc9150-c444-428f-a456-5d01e1bd67a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:36.009561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc9150-c444-428f-a456-5d01e1bd67a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:36.010195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc9150-c444-428f-a456-5d01e1bd67a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:36.011847 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc9150-c444-428f-a456-5d01e1bd67a2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:36.023347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1260/2496] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:21:36.026988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc9150-c444-428f-a456-5d01e1bd67a2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:36.037984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc9150-c444-428f-a456-5d01e1bd67a2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:36.044902 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bd7b7-7c64-42d0-b740-0705ecb6b158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:36.045540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bd7b7-7c64-42d0-b740-0705ecb6b158 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:36.045771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bd7b7-7c64-42d0-b740-0705ecb6b158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:36.046046 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bd7b7-7c64-42d0-b740-0705ecb6b158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:36.046676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bd7b7-7c64-42d0-b740-0705ecb6b158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:36.046815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1237/2497] 10.222.0.58 () {58 vars in 967 bytes} [Tue Aug 23 08:21:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:21:36.048512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bd7b7-7c64-42d0-b740-0705ecb6b158 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:36.062257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9968f9-b2c4-4204-9c0b-a25670186f10 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:36.063794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6d9968f9-b2c4-4204-9c0b-a25670186f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:36.064007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1261/2498] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:21:36.068593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bd7b7-7c64-42d0-b740-0705ecb6b158 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:36.091779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bd7b7-7c64-42d0-b740-0705ecb6b158 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:36.098659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:36.099592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:36.099876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:36.100575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] PATH_INFO: `/v3/projects/d6382332279746739a247cfb7085cebf/users/6a05807b7fd04df38e3d1fa455ec2384/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:36.120307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6382332279746739a247cfb7085cebf, user_id=6a05807b7fd04df38e3d1fa455ec2384, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:36.123166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:36.170758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9968f9-b2c4-4204-9c0b-a25670186f10 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:36.176164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d3f60-70ef-4484-91e8-9e47157e1970 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:36.176768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3f60-70ef-4484-91e8-9e47157e1970 service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:36.181602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3f60-70ef-4484-91e8-9e47157e1970 service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:36.181602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3f60-70ef-4484-91e8-9e47157e1970 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:36.187221 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d3f60-70ef-4484-91e8-9e47157e1970 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:36.189047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d3f60-70ef-4484-91e8-9e47157e1970 service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:36.200121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1238/2499] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:21:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 08:21:36.202385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1262/2500] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:21:36 2022] PUT /identity/v3/projects/d6382332279746739a247cfb7085cebf/users/6a05807b7fd04df38e3d1fa455ec2384/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:21:36.211228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bb03d-1ba0-4613-b147-5879cb78bd30 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:36.211228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bb03d-1ba0-4613-b147-5879cb78bd30 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:36.211228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bb03d-1ba0-4613-b147-5879cb78bd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:36.216638 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:36.230529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-ab6bb03d-1ba0-4613-b147-5879cb78bd30 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:36.231789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ab6bb03d-1ba0-4613-b147-5879cb78bd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:36.297776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1239/2501] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:21:36.308749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:36.310486 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:36.371969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14f169-a77a-420f-bc57-d08e92ea5c31 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:36.375718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e989ed-6fb0-40c5-9df7-d273f86969d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:36.376320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e989ed-6fb0-40c5-9df7-d273f86969d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:36.376601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e989ed-6fb0-40c5-9df7-d273f86969d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:36.376872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e989ed-6fb0-40c5-9df7-d273f86969d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:36.385999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e989ed-6fb0-40c5-9df7-d273f86969d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:36.387483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e989ed-6fb0-40c5-9df7-d273f86969d1 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:36.397800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1263/2502] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:21:37.215940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1e80d-efb1-4ad7-a526-7d005fdb3a61 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:37.216440 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1e80d-efb1-4ad7-a526-7d005fdb3a61 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:37.216563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1e80d-efb1-4ad7-a526-7d005fdb3a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:37.242509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-00e1e80d-efb1-4ad7-a526-7d005fdb3a61 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:37.243970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-00e1e80d-efb1-4ad7-a526-7d005fdb3a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:37.313974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1240/2503] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:21:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:21:37.328200 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e989ed-6fb0-40c5-9df7-d273f86969d1 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:37.423678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e989ed-6fb0-40c5-9df7-d273f86969d1 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:37.429012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf4b5f-f59d-4408-bb29-897ee8f8aa5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:37.430161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf4b5f-f59d-4408-bb29-897ee8f8aa5d service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:37.430439 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf4b5f-f59d-4408-bb29-897ee8f8aa5d service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:37.430707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf4b5f-f59d-4408-bb29-897ee8f8aa5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:37.439918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf4b5f-f59d-4408-bb29-897ee8f8aa5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:37.442039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf4b5f-f59d-4408-bb29-897ee8f8aa5d service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:37.452724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1264/2504] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:21:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:21:39.161349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1e80d-efb1-4ad7-a526-7d005fdb3a61 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:39.163056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-00e1e80d-efb1-4ad7-a526-7d005fdb3a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:39.191252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1e80d-efb1-4ad7-a526-7d005fdb3a61 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:39.201302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:39.201955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:39.202231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:39.202502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] PATH_INFO: `/v3/users/fe286cbe88934741822be3f74870b302` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:39.204644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe286cbe88934741822be3f74870b302)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:39.207668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:39.458048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe286cbe88934741822be3f74870b302'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:39.459412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe286cbe88934741822be3f74870b302'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:39.471675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe286cbe88934741822be3f74870b302'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:39.472450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe286cbe88934741822be3f74870b302'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:39.567879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe286cbe88934741822be3f74870b302'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:39.594630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1241/2505] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:21:39 2022] DELETE /identity/v3/users/fe286cbe88934741822be3f74870b302 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:21:39.599655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bfd8d-03f9-4aa0-a7e7-5639998af744 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:39.600692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bfd8d-03f9-4aa0-a7e7-5639998af744 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:39.601182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bfd8d-03f9-4aa0-a7e7-5639998af744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:39.636044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f04bfd8d-03f9-4aa0-a7e7-5639998af744 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:39.637570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f04bfd8d-03f9-4aa0-a7e7-5639998af744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:39.654663 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:39.733536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1265/2506] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:39.746091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:39.747576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:39.817614 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:39.873568 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0eb8c-9689-4de6-9569-292e735b6cc7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:39.880293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b12cfb-3806-4474-b86f-4e82571d0f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:39.883372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b12cfb-3806-4474-b86f-4e82571d0f50 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:39.883372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b12cfb-3806-4474-b86f-4e82571d0f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:39.883372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b12cfb-3806-4474-b86f-4e82571d0f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:39.903239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b12cfb-3806-4474-b86f-4e82571d0f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:39.906856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b12cfb-3806-4474-b86f-4e82571d0f50 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:39.935364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1242/2507] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:21:40.318588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bfd8d-03f9-4aa0-a7e7-5639998af744 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:40.320769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f04bfd8d-03f9-4aa0-a7e7-5639998af744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:40.337754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bfd8d-03f9-4aa0-a7e7-5639998af744 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:40.357947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:40.357947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:40.357947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:40.358242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] PATH_INFO: `/v3/projects/1e8586b18aaa47f598a11b272031217a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:40.360361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8586b18aaa47f598a11b272031217a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:40.362972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:40.550934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8586b18aaa47f598a11b272031217a', 'request_id': 'req-a9d7bebc-a503-4c22-b7ca-05135d494176'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:40.624474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8586b18aaa47f598a11b272031217a', 'request_id': 'req-a9d7bebc-a503-4c22-b7ca-05135d494176'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:40.632033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8586b18aaa47f598a11b272031217a', 'request_id': 'req-a9d7bebc-a503-4c22-b7ca-05135d494176'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:40.668252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] The token cache is being invalidate because project 1e8586b18aaa47f598a11b272031217a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:21:40.669802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:40.669802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:21:40.672089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1266/2508] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:21:40 2022] DELETE /identity/v3/projects/1e8586b18aaa47f598a11b272031217a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:21:43.441382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e10c5-9c89-48a9-b780-2b1dc7d33aaf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:43.443099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e10c5-9c89-48a9-b780-2b1dc7d33aaf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:43.443540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e10c5-9c89-48a9-b780-2b1dc7d33aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:43.512002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3c2e10c5-9c89-48a9-b780-2b1dc7d33aaf None None] MFA Rules not processed for user `6a05807b7fd04df38e3d1fa455ec2384`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:43.513914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3c2e10c5-9c89-48a9-b780-2b1dc7d33aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:43.606984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1243/2509] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:43.611631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:43.613841 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:43.737575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7bebc-a503-4c22-b7ca-05135d494176 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:43.745521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa9692-6a71-48a4-98b1-303b101eb68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:43.746700 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa9692-6a71-48a4-98b1-303b101eb68b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:43.747095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa9692-6a71-48a4-98b1-303b101eb68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:43.747951 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa9692-6a71-48a4-98b1-303b101eb68b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:43.748885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa9692-6a71-48a4-98b1-303b101eb68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:43.754052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa9692-6a71-48a4-98b1-303b101eb68b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:43.932296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1267/2510] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:21:43 2022] POST /identity/v3/projects => generated 383 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:21:43.936494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e10c5-9c89-48a9-b780-2b1dc7d33aaf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:43.972145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e10c5-9c89-48a9-b780-2b1dc7d33aaf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:44.100460 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9d2fe-e140-4d63-b179-b43a6bd32b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:44.101653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9d2fe-e140-4d63-b179-b43a6bd32b3c admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:44.102077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9d2fe-e140-4d63-b179-b43a6bd32b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:44.102355 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9d2fe-e140-4d63-b179-b43a6bd32b3c admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:44.103229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9d2fe-e140-4d63-b179-b43a6bd32b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:44.105181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9d2fe-e140-4d63-b179-b43a6bd32b3c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:44.257982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1244/2511] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:21:43 2022] POST /identity/v3/users => generated 298 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:21:44.262212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa9692-6a71-48a4-98b1-303b101eb68b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:44.279470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa9692-6a71-48a4-98b1-303b101eb68b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:44.290551 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bfb23-1f3a-470a-a327-2d6b5d13ab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:44.292649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bfb23-1f3a-470a-a327-2d6b5d13ab5b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:44.293861 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bfb23-1f3a-470a-a327-2d6b5d13ab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:44.294391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bfb23-1f3a-470a-a327-2d6b5d13ab5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:44.295811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bfb23-1f3a-470a-a327-2d6b5d13ab5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:44.298444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bfb23-1f3a-470a-a327-2d6b5d13ab5b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:44.310401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1268/2512] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:21:44.314887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9d2fe-e140-4d63-b179-b43a6bd32b3c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:44.328690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9d2fe-e140-4d63-b179-b43a6bd32b3c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:44.333828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb5b5f-f4d7-4a1f-b99a-74fb8e103983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:44.334483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5b5f-f4d7-4a1f-b99a-74fb8e103983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:44.334764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5b5f-f4d7-4a1f-b99a-74fb8e103983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:44.335703 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5b5f-f4d7-4a1f-b99a-74fb8e103983 admin admin] PATH_INFO: `/v3/projects/510666e8d81e4a62a2526b95855eb05c/users/6b0be4223e814c4bb46370cf42cbe553/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:44.353277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb5b5f-f4d7-4a1f-b99a-74fb8e103983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510666e8d81e4a62a2526b95855eb05c, user_id=6b0be4223e814c4bb46370cf42cbe553, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:44.356540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb5b5f-f4d7-4a1f-b99a-74fb8e103983 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:44.452424 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1245/2513] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:21:44 2022] PUT /identity/v3/projects/510666e8d81e4a62a2526b95855eb05c/users/6b0be4223e814c4bb46370cf42cbe553/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:21:44.459090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bfb23-1f3a-470a-a327-2d6b5d13ab5b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:44.477190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bfb23-1f3a-470a-a327-2d6b5d13ab5b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:44.499834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:44.577113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b407ec-dfa7-4a0c-b1c1-d1dabe117930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:44.578082 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b407ec-dfa7-4a0c-b1c1-d1dabe117930 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:44.578781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b407ec-dfa7-4a0c-b1c1-d1dabe117930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:44.579168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b407ec-dfa7-4a0c-b1c1-d1dabe117930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:44.580101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b407ec-dfa7-4a0c-b1c1-d1dabe117930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:44.582732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b407ec-dfa7-4a0c-b1c1-d1dabe117930 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:44.599020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1269/2514] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:21:44.601267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb5b5f-f4d7-4a1f-b99a-74fb8e103983 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:44.614296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb5b5f-f4d7-4a1f-b99a-74fb8e103983 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:44.620144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:44.620804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:44.621084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:44.621367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] PATH_INFO: `/v3/projects/510666e8d81e4a62a2526b95855eb05c/users/6b0be4223e814c4bb46370cf42cbe553/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:44.623447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] RBAC: Authorizing `identity:create_grant(project_id=510666e8d81e4a62a2526b95855eb05c, user_id=6b0be4223e814c4bb46370cf42cbe553, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:44.626879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:44.638780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1246/2515] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:21:44 2022] PUT /identity/v3/projects/510666e8d81e4a62a2526b95855eb05c/users/6b0be4223e814c4bb46370cf42cbe553/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:21:52.323219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8332c4-a12d-483c-980f-2812b61cb59b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:52.323671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8332c4-a12d-483c-980f-2812b61cb59b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:52.323790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8332c4-a12d-483c-980f-2812b61cb59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:52.354547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8c8332c4-a12d-483c-980f-2812b61cb59b None None] MFA Rules not processed for user `6b0be4223e814c4bb46370cf42cbe553`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:52.356279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8c8332c4-a12d-483c-980f-2812b61cb59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:52.439064 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1270/2516] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:21:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:52.451569 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:52.453326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:52.463439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:52.512186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb08c6a-208f-487b-ba99-634197dc450a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:52.515846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e9526306889402c83f2d33b340c2dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:52.516471 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:52.516778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:52.517049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:52.587613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:52.590534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:52.601901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1247/2517] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:21:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:21:52.901774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c0035-6fbe-401c-a6fc-00d8b48c7412 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:52.902230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c0035-6fbe-401c-a6fc-00d8b48c7412 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:52.902514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c0035-6fbe-401c-a6fc-00d8b48c7412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:52.925442 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a03c0035-6fbe-401c-a6fc-00d8b48c7412 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:21:52.927224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a03c0035-6fbe-401c-a6fc-00d8b48c7412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:52.990780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1271/2518] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:21:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:21:53.006735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:53.008504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:53.085111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdafd3-6f74-475f-85ca-16b59332e06b service glance] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:53.090634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3993e2-dd08-40a0-92cd-5b7d4be1f5ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:53.091345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3993e2-dd08-40a0-92cd-5b7d4be1f5ea service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:53.091695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3993e2-dd08-40a0-92cd-5b7d4be1f5ea service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:53.092043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3993e2-dd08-40a0-92cd-5b7d4be1f5ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:53.102923 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3993e2-dd08-40a0-92cd-5b7d4be1f5ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:53.105844 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3993e2-dd08-40a0-92cd-5b7d4be1f5ea service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:53.116051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1248/2519] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:21:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 08:21:57.046987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c0035-6fbe-401c-a6fc-00d8b48c7412 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:57.048415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a03c0035-6fbe-401c-a6fc-00d8b48c7412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:57.095391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:21:57.105608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c0035-6fbe-401c-a6fc-00d8b48c7412 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:57.109149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fdb65-a370-4ab0-b6e4-b0b32ed55ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:57.109751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fdb65-a370-4ab0-b6e4-b0b32ed55ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:57.110026 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fdb65-a370-4ab0-b6e4-b0b32ed55ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:57.110290 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fdb65-a370-4ab0-b6e4-b0b32ed55ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:57.112230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-362fdb65-a370-4ab0-b6e4-b0b32ed55ac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:57.185807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fdb65-a370-4ab0-b6e4-b0b32ed55ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:57.187297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fdb65-a370-4ab0-b6e4-b0b32ed55ac2 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:57.204024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1272/2520] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:21:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:21:58.480297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3993e2-dd08-40a0-92cd-5b7d4be1f5ea service swift] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:21:58.491751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3993e2-dd08-40a0-92cd-5b7d4be1f5ea service swift] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:21:58.496039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18204241-9c2a-4026-91d8-d028d65074a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:21:58.496629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18204241-9c2a-4026-91d8-d028d65074a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:21:58.496903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18204241-9c2a-4026-91d8-d028d65074a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:21:58.497169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18204241-9c2a-4026-91d8-d028d65074a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:21:58.499294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-18204241-9c2a-4026-91d8-d028d65074a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:21:58.514511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18204241-9c2a-4026-91d8-d028d65074a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:21:58.516380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18204241-9c2a-4026-91d8-d028d65074a0 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:21:58.528850 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1249/2521] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:21:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:00.970222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acb2af-90c1-41bf-a878-a1105b2b5898 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:00.970664 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acb2af-90c1-41bf-a878-a1105b2b5898 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:00.970793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acb2af-90c1-41bf-a878-a1105b2b5898 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:00.972307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1273/2522] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:22:00.983477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45049c9b-b665-4c9c-8fcb-1598da965362 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:00.983477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45049c9b-b665-4c9c-8fcb-1598da965362 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:00.983477 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45049c9b-b665-4c9c-8fcb-1598da965362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:01.020110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-45049c9b-b665-4c9c-8fcb-1598da965362 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:22:01.021645 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-45049c9b-b665-4c9c-8fcb-1598da965362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:01.032078 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1250/2523] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:22:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:22:01.047958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acb2af-90c1-41bf-a878-a1105b2b5898 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:01.049769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-53acb2af-90c1-41bf-a878-a1105b2b5898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:01.066721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acb2af-90c1-41bf-a878-a1105b2b5898 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:01.072312 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7dcdd-6a16-471e-aad7-dddaf2d5a444 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e9526306889402c83f2d33b340c2dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:01.072917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7dcdd-6a16-471e-aad7-dddaf2d5a444 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:01.073191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7dcdd-6a16-471e-aad7-dddaf2d5a444 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:01.073470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7dcdd-6a16-471e-aad7-dddaf2d5a444 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:01.091981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7dcdd-6a16-471e-aad7-dddaf2d5a444 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:01.093759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7dcdd-6a16-471e-aad7-dddaf2d5a444 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:01.105804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1274/2524] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:22:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:01.377732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe859c-1f19-438f-865d-288aabc20ece None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:01.377972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe859c-1f19-438f-865d-288aabc20ece None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:01.378245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe859c-1f19-438f-865d-288aabc20ece None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:01.379873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1251/2525] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:22:01.384938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a838-002e-40b2-a3c4-50bd66a39ac5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:01.385318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a838-002e-40b2-a3c4-50bd66a39ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:01.385689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a838-002e-40b2-a3c4-50bd66a39ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:01.411656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8e67a838-002e-40b2-a3c4-50bd66a39ac5 None None] MFA Rules not processed for user `d99632164b3b4a1f95736a65da217db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:22:01.414170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8e67a838-002e-40b2-a3c4-50bd66a39ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:01.421423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1275/2526] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:22:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:22:01.441999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe859c-1f19-438f-865d-288aabc20ece None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:01.472891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe859c-1f19-438f-865d-288aabc20ece None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:01.479146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1f7d-9d2f-49af-b5e8-eb2335e740b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:01.479945 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1f7d-9d2f-49af-b5e8-eb2335e740b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:01.480494 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1f7d-9d2f-49af-b5e8-eb2335e740b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:01.481716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1f7d-9d2f-49af-b5e8-eb2335e740b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:01.492359 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab1f7d-9d2f-49af-b5e8-eb2335e740b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:01.493821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab1f7d-9d2f-49af-b5e8-eb2335e740b2 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:01.502868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1252/2527] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:22:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:01.606992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaa4df-07f7-468d-93c0-583dcc8b5d90 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:01.608072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaa4df-07f7-468d-93c0-583dcc8b5d90 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:01.608684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaa4df-07f7-468d-93c0-583dcc8b5d90 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:01.610395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1276/2528] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:22:01.616377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9316-d3ba-4ee0-9b0d-85db61e96aa3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:01.616668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9316-d3ba-4ee0-9b0d-85db61e96aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:01.616943 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9316-d3ba-4ee0-9b0d-85db61e96aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:01.638932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3a2e9316-d3ba-4ee0-9b0d-85db61e96aa3 None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:22:01.640383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3a2e9316-d3ba-4ee0-9b0d-85db61e96aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:01.650957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1253/2529] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:22:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:22:01.665329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaa4df-07f7-468d-93c0-583dcc8b5d90 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:01.667212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a2aaa4df-07f7-468d-93c0-583dcc8b5d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:01.683362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaa4df-07f7-468d-93c0-583dcc8b5d90 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:01.687084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:01.687685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:01.687989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:01.688261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:01.703050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:01.704755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:01.720616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1277/2530] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:22:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:04.975558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972d7c4-91c0-4e68-9750-34ff25a1e132 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:04.977981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972d7c4-91c0-4e68-9750-34ff25a1e132 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:04.977981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972d7c4-91c0-4e68-9750-34ff25a1e132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:04.994505 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e972d7c4-91c0-4e68-9750-34ff25a1e132 None None] MFA Rules not processed for user `6a05807b7fd04df38e3d1fa455ec2384`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:22:04.995746 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e972d7c4-91c0-4e68-9750-34ff25a1e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:05.002486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1254/2531] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:22:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:22:10.321903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:10.323440 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:10.337964 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b80d6-63fc-4b06-858f-aa224d1754e4 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:10.343427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:10.350609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:10.350609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:10.350609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:10.350609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:10.404482 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:10.406107 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:10.415325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1278/2532] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:22:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:12.554129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a56ad-9621-45ad-818d-3b75444ab151 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:12.554618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a56ad-9621-45ad-818d-3b75444ab151 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:12.554734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a56ad-9621-45ad-818d-3b75444ab151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:12.563735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:22:12.575789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-609a56ad-9621-45ad-818d-3b75444ab151 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:22:12.577011 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-609a56ad-9621-45ad-818d-3b75444ab151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:12.584955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1255/2533] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:22:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:22:12.601499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:12.608243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec03b4-922c-4831-98f3-1a77d77b0fc9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:12.608924 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec03b4-922c-4831-98f3-1a77d77b0fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:12.609417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec03b4-922c-4831-98f3-1a77d77b0fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:12.615280 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18304ca-e7d6-4e06-bfb9-5b0ff37c550b service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:12.622687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e535c5-1feb-4f28-8515-8a24e11462f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:12.622854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e535c5-1feb-4f28-8515-8a24e11462f3 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:12.623219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e535c5-1feb-4f28-8515-8a24e11462f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:12.623736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e535c5-1feb-4f28-8515-8a24e11462f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:12.634699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e535c5-1feb-4f28-8515-8a24e11462f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:12.635066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-eaec03b4-922c-4831-98f3-1a77d77b0fc9 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:22:12.636335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-eaec03b4-922c-4831-98f3-1a77d77b0fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:12.636453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e535c5-1feb-4f28-8515-8a24e11462f3 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:12.644882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1256/2534] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:22:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:22:12.649125 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1279/2535] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:22:12.671298 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec03b4-922c-4831-98f3-1a77d77b0fc9 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:12.688375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec03b4-922c-4831-98f3-1a77d77b0fc9 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:12.694932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e85fe-ae03-4ba7-9436-63bb4aafbe4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:12.702344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e85fe-ae03-4ba7-9436-63bb4aafbe4d service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:12.702344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e85fe-ae03-4ba7-9436-63bb4aafbe4d service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:12.702344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e85fe-ae03-4ba7-9436-63bb4aafbe4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:12.705175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e85fe-ae03-4ba7-9436-63bb4aafbe4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:12.706807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e85fe-ae03-4ba7-9436-63bb4aafbe4d service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:12.716841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1257/2536] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:22:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:22:22.643273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=83 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:22:23.037595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=10 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:22:45.637950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e535c5-1feb-4f28-8515-8a24e11462f3 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:45.648075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e535c5-1feb-4f28-8515-8a24e11462f3 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:45.651833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:45.652451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:45.652774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:45.653048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:45.654844 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:45.664586 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:45.666007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:45.674368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1280/2537] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:22:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:46.375604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e85fe-ae03-4ba7-9436-63bb4aafbe4d service swift] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:46.389157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e85fe-ae03-4ba7-9436-63bb4aafbe4d service swift] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:46.395415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:46.396013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:46.396758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:46.396758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:46.398631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:46.411957 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:46.413576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:46.426522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1258/2538] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:22:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:47.914650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:47.916795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:47.934633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26589-3167-4d96-b66f-fbd86234fac2 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:47.943764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb34bd1-162a-42ca-b9f5-20b05605f296 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:47.944638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb34bd1-162a-42ca-b9f5-20b05605f296 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:47.945076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb34bd1-162a-42ca-b9f5-20b05605f296 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:47.945477 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb34bd1-162a-42ca-b9f5-20b05605f296 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:47.947305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-adb34bd1-162a-42ca-b9f5-20b05605f296 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:48.015794 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb34bd1-162a-42ca-b9f5-20b05605f296 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:48.017491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb34bd1-162a-42ca-b9f5-20b05605f296 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:48.028165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1281/2539] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:22:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:55.669374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:22:55.683504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e9fde-3a20-4969-8b5e-1aae5414711b service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:22:55.689430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:22:55.690295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:22:55.690574 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:22:55.690843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:22:55.692586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:22:55.705901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:22:55.707385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:22:55.718410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1259/2540] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:22:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:22:58.023068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=34 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:23:05.713652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=12 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:23:06.443526 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1fac2-f922-41e8-b9f2-5a2d13d55438 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:23:06.444002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1fac2-f922-41e8-b9f2-5a2d13d55438 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:23:06.444100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1fac2-f922-41e8-b9f2-5a2d13d55438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:23:06.483286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-efc1fac2-f922-41e8-b9f2-5a2d13d55438 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:23:06.483839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-efc1fac2-f922-41e8-b9f2-5a2d13d55438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:23:06.516319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1282/2541] 10.222.0.58 () {58 vars in 967 bytes} [Tue Aug 23 08:23:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:23:06.548370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:23:06.561654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3794c-8158-4950-a5f9-0a5883ac8dab service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:23:06.570842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ad036-62aa-4e6f-a643-fede3920e6c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:23:06.571495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ad036-62aa-4e6f-a643-fede3920e6c4 service swift] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:23:06.571808 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ad036-62aa-4e6f-a643-fede3920e6c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:23:06.572141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ad036-62aa-4e6f-a643-fede3920e6c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:23:06.583670 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ad036-62aa-4e6f-a643-fede3920e6c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:23:06.585410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ad036-62aa-4e6f-a643-fede3920e6c4 service swift] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:23:06.594577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1260/2542] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:23:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:23:11.679500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac3459-5d3a-497e-8e39-bf17743f2498 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:23:11.680039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac3459-5d3a-497e-8e39-bf17743f2498 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:23:11.680039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac3459-5d3a-497e-8e39-bf17743f2498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:23:11.699672 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-96ac3459-5d3a-497e-8e39-bf17743f2498 None None] MFA Rules not processed for user `a0a59ef94e054c3d8175e1c0f1ad783a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:23:11.700872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-96ac3459-5d3a-497e-8e39-bf17743f2498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:23:11.756553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1283/2543] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:23:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:23:12.019777 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ad036-62aa-4e6f-a643-fede3920e6c4 service swift] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:23:12.030396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ad036-62aa-4e6f-a643-fede3920e6c4 service swift] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:23:12.034600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:23:12.035301 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:23:12.035581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:23:12.035851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:23:12.037627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:23:12.049069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:23:12.050504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:23:12.058897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1261/2544] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:23:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:23:12.624968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac3459-5d3a-497e-8e39-bf17743f2498 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:23:12.626525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-96ac3459-5d3a-497e-8e39-bf17743f2498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:23:12.709230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac3459-5d3a-497e-8e39-bf17743f2498 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:23:12.715678 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a537306-03ff-46f8-a907-e0c21aa9fbef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:23:12.716227 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a537306-03ff-46f8-a907-e0c21aa9fbef service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:23:12.716506 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a537306-03ff-46f8-a907-e0c21aa9fbef service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:23:12.716808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a537306-03ff-46f8-a907-e0c21aa9fbef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:23:12.718582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5a537306-03ff-46f8-a907-e0c21aa9fbef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:23:12.730748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a537306-03ff-46f8-a907-e0c21aa9fbef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:23:12.732502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a537306-03ff-46f8-a907-e0c21aa9fbef service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:23:12.743122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1284/2545] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:23:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:23:20.154918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:23:20.163567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5cdfa-150d-4f0a-a259-78116b67c346 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:23:20.167665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7fccc-a509-4388-a352-106ddee5d514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:23:20.168255 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fccc-a509-4388-a352-106ddee5d514 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:23:20.168531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fccc-a509-4388-a352-106ddee5d514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:23:20.168802 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fccc-a509-4388-a352-106ddee5d514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:23:20.177562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7fccc-a509-4388-a352-106ddee5d514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:23:20.179017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7fccc-a509-4388-a352-106ddee5d514 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:23:20.188601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1262/2546] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:23:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:23:22.738344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=60 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:23:30.184166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=18 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:23:37.119237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09aa68-b340-4ba6-8959-9b16cadbbb03 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:23:37.119735 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09aa68-b340-4ba6-8959-9b16cadbbb03 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:23:37.119834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09aa68-b340-4ba6-8959-9b16cadbbb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:23:37.147790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cf09aa68-b340-4ba6-8959-9b16cadbbb03 None None] MFA Rules not processed for user `b3c984088c0448f8abc0a0b60b820b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:23:37.149013 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cf09aa68-b340-4ba6-8959-9b16cadbbb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:23:37.172305 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1285/2547] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:23:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:23:47.170253 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=15 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:24:34.689736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7fccc-a509-4388-a352-106ddee5d514 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:34.733261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7fccc-a509-4388-a352-106ddee5d514 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:34.738370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:34.738995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:34.739334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:34.739616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:34.741327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:34.762408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:34.764035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:34.773934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1263/2548] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:24:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:24:34.863231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09aa68-b340-4ba6-8959-9b16cadbbb03 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:34.864661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cf09aa68-b340-4ba6-8959-9b16cadbbb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:34.962056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09aa68-b340-4ba6-8959-9b16cadbbb03 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:34.966757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:34.967418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:34.967697 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:34.967971 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] PATH_INFO: `/v3/users/a0a59ef94e054c3d8175e1c0f1ad783a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:34.969099 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a59ef94e054c3d8175e1c0f1ad783a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:34.971092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:35.252668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a59ef94e054c3d8175e1c0f1ad783a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.259537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a59ef94e054c3d8175e1c0f1ad783a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.260233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a59ef94e054c3d8175e1c0f1ad783a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.260849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a59ef94e054c3d8175e1c0f1ad783a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.273366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9168b01b-6842-4e27-b57e-37051811e690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a59ef94e054c3d8175e1c0f1ad783a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.341738 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1286/2549] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:24:34 2022] DELETE /identity/v3/users/a0a59ef94e054c3d8175e1c0f1ad783a => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:24:35.618205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:35.631943 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127fdf5-a45f-4b45-946c-a3945033baab service placement] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:35.686551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:35.687219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:35.687502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:35.687776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] PATH_INFO: `/v3/projects/ee175b1f398844f6be0075d3e127e8b3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:35.688888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee175b1f398844f6be0075d3e127e8b3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:35.690806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:35.787634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee175b1f398844f6be0075d3e127e8b3', 'request_id': 'req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.793627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee175b1f398844f6be0075d3e127e8b3', 'request_id': 'req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.799716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee175b1f398844f6be0075d3e127e8b3', 'request_id': 'req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.874121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] The token cache is being invalidate because project ee175b1f398844f6be0075d3e127e8b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:24:35.874335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.874987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ba8eb858-9558-4d8b-b460-f8b4cc2a8c28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:24:35.876590 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1264/2550] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:24:35 2022] DELETE /identity/v3/projects/ee175b1f398844f6be0075d3e127e8b3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:24:35.881705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3be09-c1c8-40d0-a827-f8e9a4920b15 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:35.882171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3be09-c1c8-40d0-a827-f8e9a4920b15 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:35.882335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3be09-c1c8-40d0-a827-f8e9a4920b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:35.939760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0ed3be09-c1c8-40d0-a827-f8e9a4920b15 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:24:35.941009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0ed3be09-c1c8-40d0-a827-f8e9a4920b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:35.953975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:24:35.999397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1287/2551] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:24:36.007259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c001c9-5036-452c-b7a4-6ee61c537fb4 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.007526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c001c9-5036-452c-b7a4-6ee61c537fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.007794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c001c9-5036-452c-b7a4-6ee61c537fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.028003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-26c001c9-5036-452c-b7a4-6ee61c537fb4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:24:36.029190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-26c001c9-5036-452c-b7a4-6ee61c537fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:36.037076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1265/2552] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:24:36.041205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3be09-c1c8-40d0-a827-f8e9a4920b15 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:36.051767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3be09-c1c8-40d0-a827-f8e9a4920b15 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:36.056458 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:36.057077 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.057406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.057680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.058356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:24:36.058669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:36.060304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:36.072024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1288/2553] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:24:36.077504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c001c9-5036-452c-b7a4-6ee61c537fb4 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:36.086718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c001c9-5036-452c-b7a4-6ee61c537fb4 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:36.091889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc56c14-ab5a-4e0d-8248-484ed1dbcdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:36.092469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc56c14-ab5a-4e0d-8248-484ed1dbcdf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.092743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc56c14-ab5a-4e0d-8248-484ed1dbcdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.093017 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc56c14-ab5a-4e0d-8248-484ed1dbcdf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.093627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc56c14-ab5a-4e0d-8248-484ed1dbcdf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:36.095563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc56c14-ab5a-4e0d-8248-484ed1dbcdf5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:36.171978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:24:36.178061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1266/2554] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:24:36 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:24:36.182236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:36.193169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afb2b1-3390-477c-a415-4990ddd1ce77 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:36.249856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea61f8-f37f-48c8-8e26-eab32e4b1234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:36.250487 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea61f8-f37f-48c8-8e26-eab32e4b1234 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.250796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea61f8-f37f-48c8-8e26-eab32e4b1234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.251100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea61f8-f37f-48c8-8e26-eab32e4b1234 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.257934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea61f8-f37f-48c8-8e26-eab32e4b1234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:36.257934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea61f8-f37f-48c8-8e26-eab32e4b1234 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:36.370515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1289/2555] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:24:36 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:24:36.375211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc56c14-ab5a-4e0d-8248-484ed1dbcdf5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:36.385106 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc56c14-ab5a-4e0d-8248-484ed1dbcdf5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:36.389785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0da5d-35d6-4eae-96ff-94f92ed35aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:36.391765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0da5d-35d6-4eae-96ff-94f92ed35aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.391765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0da5d-35d6-4eae-96ff-94f92ed35aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.391765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0da5d-35d6-4eae-96ff-94f92ed35aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.391765 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0da5d-35d6-4eae-96ff-94f92ed35aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:36.393095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0da5d-35d6-4eae-96ff-94f92ed35aa6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:36.402636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1267/2556] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:24:36.406423 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea61f8-f37f-48c8-8e26-eab32e4b1234 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:36.416464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea61f8-f37f-48c8-8e26-eab32e4b1234 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:36.421718 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fff978-a39f-4c11-9ad9-3c56c495c564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:36.422355 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff978-a39f-4c11-9ad9-3c56c495c564 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.422586 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff978-a39f-4c11-9ad9-3c56c495c564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.422859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff978-a39f-4c11-9ad9-3c56c495c564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.423530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fff978-a39f-4c11-9ad9-3c56c495c564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:36.425119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fff978-a39f-4c11-9ad9-3c56c495c564 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:36.436344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1290/2557] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:24:36.440467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0da5d-35d6-4eae-96ff-94f92ed35aa6 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:36.450781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0da5d-35d6-4eae-96ff-94f92ed35aa6 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:36.455699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:36.456370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.456650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.456955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] PATH_INFO: `/v3/projects/f80bce57227744ac9be2fda344a86e39/users/ef9753669de440489453104bbe61b449/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.471463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80bce57227744ac9be2fda344a86e39, user_id=ef9753669de440489453104bbe61b449, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:36.474123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:36.530706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1268/2558] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:24:36 2022] PUT /identity/v3/projects/f80bce57227744ac9be2fda344a86e39/users/ef9753669de440489453104bbe61b449/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:24:36.535652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1fda0-16a4-427e-b0c4-6b8b40f0f2f7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.535959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1fda0-16a4-427e-b0c4-6b8b40f0f2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.536308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1fda0-16a4-427e-b0c4-6b8b40f0f2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.556105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fcb1fda0-16a4-427e-b0c4-6b8b40f0f2f7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:24:36.557314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fcb1fda0-16a4-427e-b0c4-6b8b40f0f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:36.618193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1291/2559] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:24:36.628243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:36.629986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:36.707720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616214b-8b9b-4b95-ae4f-76df1bc2c090 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:36.711325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f98261-8558-4853-9306-fb4a2b0f8df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:36.711914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f98261-8558-4853-9306-fb4a2b0f8df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:36.712190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f98261-8558-4853-9306-fb4a2b0f8df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:36.712458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f98261-8558-4853-9306-fb4a2b0f8df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:36.721707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f98261-8558-4853-9306-fb4a2b0f8df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:36.723149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f98261-8558-4853-9306-fb4a2b0f8df6 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:36.733008 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1269/2560] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:24:42.235195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d1f9-70af-4b71-bcec-93f2a9c9c59a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:42.235638 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d1f9-70af-4b71-bcec-93f2a9c9c59a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:42.235995 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193d1f9-70af-4b71-bcec-93f2a9c9c59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:42.269712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b193d1f9-70af-4b71-bcec-93f2a9c9c59a None None] MFA Rules not processed for user `ef9753669de440489453104bbe61b449`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:24:42.271047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b193d1f9-70af-4b71-bcec-93f2a9c9c59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:42.290550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:24:42.363868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1292/2561] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:24:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:24:42.364528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f98261-8558-4853-9306-fb4a2b0f8df6 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:42.375224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f98261-8558-4853-9306-fb4a2b0f8df6 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:42.380632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e146275-4db2-4292-a217-cc9543be6861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:42.381233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146275-4db2-4292-a217-cc9543be6861 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:42.381510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146275-4db2-4292-a217-cc9543be6861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:42.381791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146275-4db2-4292-a217-cc9543be6861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:42.382410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e146275-4db2-4292-a217-cc9543be6861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:42.384352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e146275-4db2-4292-a217-cc9543be6861 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:42.500339 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1270/2562] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:24:42 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:24:42.501655 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193d1f9-70af-4b71-bcec-93f2a9c9c59a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:42.514704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193d1f9-70af-4b71-bcec-93f2a9c9c59a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:42.574049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d428b4-0bdd-4f6b-bc6b-24330389764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:42.574652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d428b4-0bdd-4f6b-bc6b-24330389764c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:42.574948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d428b4-0bdd-4f6b-bc6b-24330389764c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:42.575242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d428b4-0bdd-4f6b-bc6b-24330389764c admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:42.575860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d428b4-0bdd-4f6b-bc6b-24330389764c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:42.577776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d428b4-0bdd-4f6b-bc6b-24330389764c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:42.650032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1293/2563] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:24:42 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:24:42.654030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e146275-4db2-4292-a217-cc9543be6861 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:42.666238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e146275-4db2-4292-a217-cc9543be6861 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:42.671548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52896c98-f27b-45fe-b798-43e63d2958cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:42.672156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52896c98-f27b-45fe-b798-43e63d2958cf admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:42.672438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52896c98-f27b-45fe-b798-43e63d2958cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:42.672732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52896c98-f27b-45fe-b798-43e63d2958cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:42.680716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52896c98-f27b-45fe-b798-43e63d2958cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:42.680716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52896c98-f27b-45fe-b798-43e63d2958cf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:42.690104 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1271/2564] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:24:42.690993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d428b4-0bdd-4f6b-bc6b-24330389764c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:42.701612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d428b4-0bdd-4f6b-bc6b-24330389764c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:42.706311 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b51c7-3cec-4b90-ba0d-8b753b960fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:42.707041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b51c7-3cec-4b90-ba0d-8b753b960fde admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:42.707272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b51c7-3cec-4b90-ba0d-8b753b960fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:42.707542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b51c7-3cec-4b90-ba0d-8b753b960fde admin admin] PATH_INFO: `/v3/projects/b358a54339494b99bf299ed88ac0e06c/users/52cb54620c1c41a8ad17e15f2b3c1da7/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:42.725988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b51c7-3cec-4b90-ba0d-8b753b960fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=b358a54339494b99bf299ed88ac0e06c, user_id=52cb54620c1c41a8ad17e15f2b3c1da7, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:42.728792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b51c7-3cec-4b90-ba0d-8b753b960fde admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:42.846005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1294/2565] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:24:42 2022] PUT /identity/v3/projects/b358a54339494b99bf299ed88ac0e06c/users/52cb54620c1c41a8ad17e15f2b3c1da7/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:24:42.850131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52896c98-f27b-45fe-b798-43e63d2958cf admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:42.860946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52896c98-f27b-45fe-b798-43e63d2958cf admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:42.917035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638f852-851a-4eee-b8d5-4523e5184a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:42.917649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638f852-851a-4eee-b8d5-4523e5184a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:42.918514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638f852-851a-4eee-b8d5-4523e5184a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:42.918791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638f852-851a-4eee-b8d5-4523e5184a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:42.919472 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638f852-851a-4eee-b8d5-4523e5184a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:42.921450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638f852-851a-4eee-b8d5-4523e5184a98 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:42.931586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1272/2566] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:24:42.936107 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b51c7-3cec-4b90-ba0d-8b753b960fde admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:42.946651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b51c7-3cec-4b90-ba0d-8b753b960fde admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:42.951253 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:42.951897 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:42.952177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:42.952448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] PATH_INFO: `/v3/projects/b358a54339494b99bf299ed88ac0e06c/users/52cb54620c1c41a8ad17e15f2b3c1da7/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:42.954348 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b358a54339494b99bf299ed88ac0e06c, user_id=52cb54620c1c41a8ad17e15f2b3c1da7, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:42.957012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:42.967645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1295/2567] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:24:42 2022] PUT /identity/v3/projects/b358a54339494b99bf299ed88ac0e06c/users/52cb54620c1c41a8ad17e15f2b3c1da7/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:24:49.174973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28f40-dba5-4c2e-95f1-cc27a33c7fc0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:49.175521 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28f40-dba5-4c2e-95f1-cc27a33c7fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:49.175756 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28f40-dba5-4c2e-95f1-cc27a33c7fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:49.207410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:24:49.215796 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a8c28f40-dba5-4c2e-95f1-cc27a33c7fc0 None None] MFA Rules not processed for user `52cb54620c1c41a8ad17e15f2b3c1da7`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:24:49.217103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a8c28f40-dba5-4c2e-95f1-cc27a33c7fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:49.312202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1273/2568] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:24:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:24:49.327552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:49.331401 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:49.423588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142126a0-a065-4764-8084-3d5ce8b634c2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:49.431177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d836c2-d758-4064-8edb-4bddfa8e8722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:49.432006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d836c2-d758-4064-8edb-4bddfa8e8722 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:49.432554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d836c2-d758-4064-8edb-4bddfa8e8722 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:49.433050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d836c2-d758-4064-8edb-4bddfa8e8722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:49.438598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:24:49.507256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d836c2-d758-4064-8edb-4bddfa8e8722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:49.509148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d836c2-d758-4064-8edb-4bddfa8e8722 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:49.531023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1296/2569] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:24:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:24:52.861109 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df40171-2e8b-41ac-b82b-93b4658303e7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:52.861674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df40171-2e8b-41ac-b82b-93b4658303e7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:52.861674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df40171-2e8b-41ac-b82b-93b4658303e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:52.885270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-4df40171-2e8b-41ac-b82b-93b4658303e7 None None] MFA Rules not processed for user `ef9753669de440489453104bbe61b449`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:24:52.887344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4df40171-2e8b-41ac-b82b-93b4658303e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:52.897544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1274/2570] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:24:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:24:54.384524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dfdc2-60fe-4bab-a339-9d9c258df613 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:54.384980 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dfdc2-60fe-4bab-a339-9d9c258df613 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:54.385081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0dfdc2-60fe-4bab-a339-9d9c258df613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:54.406271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8b0dfdc2-60fe-4bab-a339-9d9c258df613 None None] MFA Rules not processed for user `ce316aa5c0b54ac99ade6cc4163ee209`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:24:54.407512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8b0dfdc2-60fe-4bab-a339-9d9c258df613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:54.475240 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1297/2571] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:24:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:24:54.798383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df40171-2e8b-41ac-b82b-93b4658303e7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:24:54.855704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df40171-2e8b-41ac-b82b-93b4658303e7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:24:54.859367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:24:54.860441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:24:54.860441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:24:54.860637 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:24:54.862341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:24:54.872641 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:24:54.874173 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:24:54.885558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1275/2572] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:24:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:25:04.465159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=50 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:25:04.880611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=58 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:25:17.268315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dfdc2-60fe-4bab-a339-9d9c258df613 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:17.306444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0dfdc2-60fe-4bab-a339-9d9c258df613 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:17.312265 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:17.312896 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:17.313177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:17.313449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:17.315276 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:17.341072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:17.342865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:17.354688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1298/2573] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:25:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:25:23.110793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:23.119636 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e462-d177-4fdd-8c7e-f21c865cd05d service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:23.123299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:23.123886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:23.124168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:23.124439 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:23.126753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:23.191191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:23.192736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:23.203013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1276/2574] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:25:27.349702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=18 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:25:27.380803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:27.382333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:27.403234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e73fd-96d7-4577-8407-4993636c405c service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:27.408171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:27.408919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:27.409207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:27.409484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] PATH_INFO: `/v3/users/b3c984088c0448f8abc0a0b60b820b3e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:27.410624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c984088c0448f8abc0a0b60b820b3e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:27.412788 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:27.585607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c984088c0448f8abc0a0b60b820b3e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:27.592296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c984088c0448f8abc0a0b60b820b3e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:27.592947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c984088c0448f8abc0a0b60b820b3e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:27.593545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c984088c0448f8abc0a0b60b820b3e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:27.605686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-881f73a0-6c50-4f24-8a32-a0aca5483a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c984088c0448f8abc0a0b60b820b3e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:27.668947 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1299/2575] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:25:27 2022] DELETE /identity/v3/users/b3c984088c0448f8abc0a0b60b820b3e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:25:27.945128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:27.958426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fa589-aa80-4cab-992c-9c8dcf257ae9 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:28.017443 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:28.018075 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.018353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.018623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] PATH_INFO: `/v3/projects/cfd891d8d1b7436fabfa2c48019c3565` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.019839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd891d8d1b7436fabfa2c48019c3565)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:28.021829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:28.153874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd891d8d1b7436fabfa2c48019c3565', 'request_id': 'req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:28.159946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd891d8d1b7436fabfa2c48019c3565', 'request_id': 'req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:28.166185 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd891d8d1b7436fabfa2c48019c3565', 'request_id': 'req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:28.291652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] The token cache is being invalidate because project cfd891d8d1b7436fabfa2c48019c3565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:25:28.291896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:28.292524 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-6563d2bb-25ba-4b2c-986d-43f31aa3d0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:25:28.294048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1277/2576] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:25:27 2022] DELETE /identity/v3/projects/cfd891d8d1b7436fabfa2c48019c3565 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:25:28.299099 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf425-0b62-4bfc-8a3a-5e388eaa4808 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.299392 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf425-0b62-4bfc-8a3a-5e388eaa4808 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.299673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf425-0b62-4bfc-8a3a-5e388eaa4808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.320517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f1ccf425-0b62-4bfc-8a3a-5e388eaa4808 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:25:28.321990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f1ccf425-0b62-4bfc-8a3a-5e388eaa4808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:28.380686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1300/2577] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:25:28.388976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8dfa3-624c-40d2-a78b-cbdb4af68102 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.389262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8dfa3-624c-40d2-a78b-cbdb4af68102 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.389535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8dfa3-624c-40d2-a78b-cbdb4af68102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.409516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-30b8dfa3-624c-40d2-a78b-cbdb4af68102 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:25:28.410714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-30b8dfa3-624c-40d2-a78b-cbdb4af68102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:28.418655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1278/2578] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:25:28.431351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf425-0b62-4bfc-8a3a-5e388eaa4808 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:28.441598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf425-0b62-4bfc-8a3a-5e388eaa4808 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:28.446522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:28.447151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.447426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.447689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.448316 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:25:28.448608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:28.450198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:28.462271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1301/2579] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:25:28.466619 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8dfa3-624c-40d2-a78b-cbdb4af68102 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:28.475951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8dfa3-624c-40d2-a78b-cbdb4af68102 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:28.480583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a097b06-f6a8-40bd-ac15-64197d78e157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:28.481162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097b06-f6a8-40bd-ac15-64197d78e157 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.481434 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097b06-f6a8-40bd-ac15-64197d78e157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.481701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097b06-f6a8-40bd-ac15-64197d78e157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.482320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a097b06-f6a8-40bd-ac15-64197d78e157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:28.484225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a097b06-f6a8-40bd-ac15-64197d78e157 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:28.500080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:25:28.665881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1279/2580] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:25:28 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:25:28.669919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:28.680279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de047f-30ad-4a7b-8c63-4462a6135171 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:28.735798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8e64c-4b53-47fa-826e-a97300cde52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:28.736928 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8e64c-4b53-47fa-826e-a97300cde52d admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.737209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8e64c-4b53-47fa-826e-a97300cde52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.737483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8e64c-4b53-47fa-826e-a97300cde52d admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.738138 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8e64c-4b53-47fa-826e-a97300cde52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:28.740102 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8e64c-4b53-47fa-826e-a97300cde52d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:28.747920 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:25:28.868958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1302/2581] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:25:28 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:25:28.873179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a097b06-f6a8-40bd-ac15-64197d78e157 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:28.884278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a097b06-f6a8-40bd-ac15-64197d78e157 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:28.888845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2befa3-c776-4f68-a5dc-44da293d1c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:28.889435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2befa3-c776-4f68-a5dc-44da293d1c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.889717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2befa3-c776-4f68-a5dc-44da293d1c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.889990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2befa3-c776-4f68-a5dc-44da293d1c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.890628 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2befa3-c776-4f68-a5dc-44da293d1c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:28.892273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2befa3-c776-4f68-a5dc-44da293d1c2b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:28.901944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1280/2582] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:25:28.905680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8e64c-4b53-47fa-826e-a97300cde52d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:28.915476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8e64c-4b53-47fa-826e-a97300cde52d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:28.920455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f55e3-a249-4b51-b9fa-11d60271938e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:28.921034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f55e3-a249-4b51-b9fa-11d60271938e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.921307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f55e3-a249-4b51-b9fa-11d60271938e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.921571 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f55e3-a249-4b51-b9fa-11d60271938e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.922196 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f55e3-a249-4b51-b9fa-11d60271938e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:28.924304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f55e3-a249-4b51-b9fa-11d60271938e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:28.934914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1303/2583] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:25:28.938860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2befa3-c776-4f68-a5dc-44da293d1c2b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:28.949514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2befa3-c776-4f68-a5dc-44da293d1c2b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:28.954077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:28.954727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:28.955034 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:28.955312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] PATH_INFO: `/v3/projects/6c920ebfb7b046e08503e02473294eab/users/6960cb25d98c4b02a80900d97ab34a5a/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:28.971216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c920ebfb7b046e08503e02473294eab, user_id=6960cb25d98c4b02a80900d97ab34a5a, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:28.973919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:29.046300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1281/2584] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:25:28 2022] PUT /identity/v3/projects/6c920ebfb7b046e08503e02473294eab/users/6960cb25d98c4b02a80900d97ab34a5a/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:25:29.052112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36ef72-1df7-4258-8de8-4c54ef48b859 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:29.052400 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36ef72-1df7-4258-8de8-4c54ef48b859 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:29.052676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36ef72-1df7-4258-8de8-4c54ef48b859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:29.073072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1a36ef72-1df7-4258-8de8-4c54ef48b859 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:25:29.074295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1a36ef72-1df7-4258-8de8-4c54ef48b859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:29.133954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1304/2585] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:25:29.143664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:29.145348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:29.202928 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924c52d-04b5-47d3-9ae2-2861c8213707 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:29.206624 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:29.207256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:29.207534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:29.207803 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:29.216775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:29.218190 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:29.228172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1282/2586] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:25:34.347194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7aa54-da7d-407f-ac44-308d25c8e486 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:34.347709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7aa54-da7d-407f-ac44-308d25c8e486 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:34.347709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7aa54-da7d-407f-ac44-308d25c8e486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:34.377535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d9e7aa54-da7d-407f-ac44-308d25c8e486 None None] MFA Rules not processed for user `6960cb25d98c4b02a80900d97ab34a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:25:34.378764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d9e7aa54-da7d-407f-ac44-308d25c8e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:34.458694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1305/2587] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:25:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:25:34.472377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:34.474735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:34.557330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d16a4-2c6a-443a-8938-d1ab852c53b8 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:34.563398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:34.564634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:34.564978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:34.565318 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:34.574979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:34.576683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:34.589407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1283/2588] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:25:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:25:37.219424 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a206c-7ee9-4624-9f80-ea63bb141f2b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:37.219957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a206c-7ee9-4624-9f80-ea63bb141f2b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:37.219957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a206c-7ee9-4624-9f80-ea63bb141f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:37.242872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-552a206c-7ee9-4624-9f80-ea63bb141f2b None None] MFA Rules not processed for user `6960cb25d98c4b02a80900d97ab34a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:25:37.244154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-552a206c-7ee9-4624-9f80-ea63bb141f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:37.254297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1306/2589] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:25:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:25:40.803930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:25:40.805427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:40.819878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4895f-adb3-4e44-9e8b-64af83d1b1e9 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:25:40.824005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c12ae-859c-4b2f-b276-5edfaa171dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:25:40.824683 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c12ae-859c-4b2f-b276-5edfaa171dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:25:40.824948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c12ae-859c-4b2f-b276-5edfaa171dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:25:40.825226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c12ae-859c-4b2f-b276-5edfaa171dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:25:40.827110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-480c12ae-859c-4b2f-b276-5edfaa171dc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:25:40.876475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:25:40.894088 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c12ae-859c-4b2f-b276-5edfaa171dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:25:40.895495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c12ae-859c-4b2f-b276-5edfaa171dc3 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:25:40.904779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1284/2590] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:25:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:25:47.234987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:25:51.257149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:26:09.303805 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a206c-7ee9-4624-9f80-ea63bb141f2b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:09.305226 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-552a206c-7ee9-4624-9f80-ea63bb141f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:09.396391 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a206c-7ee9-4624-9f80-ea63bb141f2b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:09.402285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:09.402881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:09.403188 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:09.403459 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:09.405159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:09.417005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:09.418719 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:09.428576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1307/2591] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:26:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:26:19.424429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=41 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:26:29.637398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0dd90-2a12-4739-891e-0ac24d5585e3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:29.637833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0dd90-2a12-4739-891e-0ac24d5585e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:29.637942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0dd90-2a12-4739-891e-0ac24d5585e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:29.666514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-11e0dd90-2a12-4739-891e-0ac24d5585e3 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:26:29.667745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-11e0dd90-2a12-4739-891e-0ac24d5585e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:29.722841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1285/2592] 10.222.0.58 () {58 vars in 967 bytes} [Tue Aug 23 08:26:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:26:29.863688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:29.865174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:29.934830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d1de3-0804-49d7-b9ac-a5ed39ce99be service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:29.939925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6131cf-1100-426d-b84f-872648677f4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:29.940530 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6131cf-1100-426d-b84f-872648677f4b service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:29.940807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6131cf-1100-426d-b84f-872648677f4b service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:29.941078 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6131cf-1100-426d-b84f-872648677f4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:29.948997 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6131cf-1100-426d-b84f-872648677f4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:29.951218 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6131cf-1100-426d-b84f-872648677f4b service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:29.959158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1308/2593] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:26:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:26:34.367823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0dd90-2a12-4739-891e-0ac24d5585e3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:34.381489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0dd90-2a12-4739-891e-0ac24d5585e3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:34.386871 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:34.387526 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:34.387804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:34.388668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:34.390149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:34.402406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:34.404061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:34.415749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1286/2594] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:26:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:26:36.128184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6131cf-1100-426d-b84f-872648677f4b service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:36.138430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6131cf-1100-426d-b84f-872648677f4b service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:36.142037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:36.142696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:36.143087 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:36.143421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:36.145257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:36.157352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:36.158873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:36.168937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1309/2595] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:26:38.244895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:38.246451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:38.260655 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e842e-5a8d-43bf-b8af-c681f21bc135 service placement] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:38.265852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:38.266475 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:38.266752 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:38.267061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] PATH_INFO: `/v3/users/6a05807b7fd04df38e3d1fa455ec2384` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:38.268176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a05807b7fd04df38e3d1fa455ec2384)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:38.270112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:38.439583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a05807b7fd04df38e3d1fa455ec2384'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:38.440235 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a05807b7fd04df38e3d1fa455ec2384'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:38.447112 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a05807b7fd04df38e3d1fa455ec2384'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:38.447737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a05807b7fd04df38e3d1fa455ec2384'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:38.505787 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a05807b7fd04df38e3d1fa455ec2384'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:38.523938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1287/2596] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:26:38 2022] DELETE /identity/v3/users/6a05807b7fd04df38e3d1fa455ec2384 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:26:38.788204 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:38.805733 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a63df-1199-482d-9645-e86c2219c766 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:38.856981 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:38.857607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:38.857885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:38.858156 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] PATH_INFO: `/v3/projects/d6382332279746739a247cfb7085cebf` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:38.859307 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6382332279746739a247cfb7085cebf)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:38.861232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:39.009937 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6382332279746739a247cfb7085cebf', 'request_id': 'req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.094890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6382332279746739a247cfb7085cebf', 'request_id': 'req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.101244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6382332279746739a247cfb7085cebf', 'request_id': 'req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.112455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] The token cache is being invalidate because project d6382332279746739a247cfb7085cebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:26:39.112743 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.113382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.114906 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1310/2597] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:26:38 2022] DELETE /identity/v3/projects/d6382332279746739a247cfb7085cebf => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:26:39.119175 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:39.120571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:39.140429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329a2f1-2c03-48f5-8fd3-331190671fd8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:39.189238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:26:39.195813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:39.196436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:39.196713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:39.196983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] PATH_INFO: `/v3/users/6b0be4223e814c4bb46370cf42cbe553` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:39.198134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b0be4223e814c4bb46370cf42cbe553)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:39.200700 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:39.383309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b0be4223e814c4bb46370cf42cbe553'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.383941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b0be4223e814c4bb46370cf42cbe553'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.391121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b0be4223e814c4bb46370cf42cbe553'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.391785 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b0be4223e814c4bb46370cf42cbe553'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.465828 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b0be4223e814c4bb46370cf42cbe553'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:39.483617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1288/2598] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:26:39 2022] DELETE /identity/v3/users/6b0be4223e814c4bb46370cf42cbe553 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:26:39.725281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:39.726867 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:39.743085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7b5fd-6f99-429c-abe0-2f7fd5493c7c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:39.747919 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:26:39.802720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:39.803427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:39.803706 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:39.803978 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] PATH_INFO: `/v3/projects/510666e8d81e4a62a2526b95855eb05c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:39.805149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510666e8d81e4a62a2526b95855eb05c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:39.807130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:39.917721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510666e8d81e4a62a2526b95855eb05c', 'request_id': 'req-7d2bc2f8-352e-4697-ae75-07341e8ee933'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:40.008105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510666e8d81e4a62a2526b95855eb05c', 'request_id': 'req-7d2bc2f8-352e-4697-ae75-07341e8ee933'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:40.013855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510666e8d81e4a62a2526b95855eb05c', 'request_id': 'req-7d2bc2f8-352e-4697-ae75-07341e8ee933'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:40.024839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] The token cache is being invalidate because project 510666e8d81e4a62a2526b95855eb05c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:26:40.025130 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:40.025776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:26:40.027304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1311/2599] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:26:39 2022] DELETE /identity/v3/projects/510666e8d81e4a62a2526b95855eb05c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:26:49.479587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=28,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:26:50.021878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:26:57.828413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:57.829894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:57.886791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369eb7-6fca-4c6f-b61c-85899d6d385c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:57.934605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95fd66-ed3d-4976-b10f-93c27c1e9713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:57.942060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fd66-ed3d-4976-b10f-93c27c1e9713 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:57.942060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fd66-ed3d-4976-b10f-93c27c1e9713 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:57.942060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95fd66-ed3d-4976-b10f-93c27c1e9713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:57.942060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4a95fd66-ed3d-4976-b10f-93c27c1e9713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:58.007712 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95fd66-ed3d-4976-b10f-93c27c1e9713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:58.009139 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95fd66-ed3d-4976-b10f-93c27c1e9713 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:58.020745 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1289/2600] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:26:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:26:59.205164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:26:59.206614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:59.218830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc2f8-352e-4697-ae75-07341e8ee933 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:26:59.222593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:26:59.223264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:26:59.223544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:26:59.223815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:26:59.225567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:26:59.239493 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:26:59.240926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:26:59.252397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1312/2601] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:26:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:27:04.885937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be75e1-36c6-49c1-9c19-7a00c9d4137a None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:04.886455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be75e1-36c6-49c1-9c19-7a00c9d4137a None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:04.886564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be75e1-36c6-49c1-9c19-7a00c9d4137a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:04.906869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-41be75e1-36c6-49c1-9c19-7a00c9d4137a None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:27:04.908117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-41be75e1-36c6-49c1-9c19-7a00c9d4137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:04.965615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1290/2602] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:27:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:27:04.977489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:27:04.978885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:05.046451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce4ac3-8ef8-46d1-8c89-3e5f1bb6117d service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:27:05.051436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:27:05.052028 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:05.052302 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:05.052567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:05.060425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:27:05.062626 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:27:05.070588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1313/2603] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:27:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:27:10.000399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be75e1-36c6-49c1-9c19-7a00c9d4137a None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:27:10.002009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-41be75e1-36c6-49c1-9c19-7a00c9d4137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:10.013226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:27:10.024070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be75e1-36c6-49c1-9c19-7a00c9d4137a None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:27:10.030919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:27:10.031757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:10.032161 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:10.032529 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:10.034374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:10.144758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:27:10.146422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:27:10.168960 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1291/2604] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:27:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:27:15.067360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=38 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:27:20.163054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=40 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:27:23.839185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:27:23.840617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:23.899370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481acf-eaaa-40e5-879a-5362a3c64c90 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:27:23.903025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e9526306889402c83f2d33b340c2dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:27:23.903632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:23.903912 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:23.904184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:23.906541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:23.921443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:27:23.922860 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:27:23.933722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1314/2605] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:27:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:27:24.176134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:27:24.185514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e68efd-8bd6-4640-9e3b-664b0a9f761b service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:27:24.189125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809353e7-22b5-4e01-bdca-265dc833e905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:27:24.194912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809353e7-22b5-4e01-bdca-265dc833e905 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:24.194912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809353e7-22b5-4e01-bdca-265dc833e905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:24.194912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809353e7-22b5-4e01-bdca-265dc833e905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:24.194912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-809353e7-22b5-4e01-bdca-265dc833e905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:24.201522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809353e7-22b5-4e01-bdca-265dc833e905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:27:24.202961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809353e7-22b5-4e01-bdca-265dc833e905 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:27:24.211047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1292/2606] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:27:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:27:24.283943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:27:24.285382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:24.297485 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dd330-fda7-4daa-a95c-7e9c85719fcd service glance] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:27:24.301037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:27:24.301622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:24.301895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:24.302160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:24.303806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:24.318317 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:27:24.319818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:27:24.331531 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1315/2607] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:27:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:27:26.912338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095198-07b8-495b-88bc-dd3769236ed5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:26.912811 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095198-07b8-495b-88bc-dd3769236ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:26.912919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095198-07b8-495b-88bc-dd3769236ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:26.961977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-79095198-07b8-495b-88bc-dd3769236ed5 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:27:26.963227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-79095198-07b8-495b-88bc-dd3769236ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:26.979537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1293/2608] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Aug 23 08:27:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:27:26.991271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:27:27.012072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92dbcc-8975-40a2-9663-25906c6760bf service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:27:27.017781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ca312-244f-4142-9e17-20f71a2b198a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:27:27.018365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ca312-244f-4142-9e17-20f71a2b198a service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:27.018640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ca312-244f-4142-9e17-20f71a2b198a service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:27.018911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ca312-244f-4142-9e17-20f71a2b198a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:27.027116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ca312-244f-4142-9e17-20f71a2b198a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:27:27.028728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ca312-244f-4142-9e17-20f71a2b198a service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:27:27.036683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1316/2609] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:27:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 08:27:36.976657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=24 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:27:37.033527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=42 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:27:45.467659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698969ca-4e96-4cdf-a381-1b86ee1e5019 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:45.468138 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698969ca-4e96-4cdf-a381-1b86ee1e5019 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:45.468243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698969ca-4e96-4cdf-a381-1b86ee1e5019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:45.487514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-698969ca-4e96-4cdf-a381-1b86ee1e5019 None None] MFA Rules not processed for user `d93e24780ff740b58285575d6efdffa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:27:45.488754 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-698969ca-4e96-4cdf-a381-1b86ee1e5019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:27:45.495922 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1294/2610] 10.222.0.58 () {58 vars in 967 bytes} [Tue Aug 23 08:27:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:27:45.519052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ca312-244f-4142-9e17-20f71a2b198a service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:27:45.529525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ca312-244f-4142-9e17-20f71a2b198a service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:27:45.534436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec59847-1943-492c-a465-5976550d4fb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4705311aa574da7b9f1b9d31c40cd72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:27:45.535047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec59847-1943-492c-a465-5976550d4fb3 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:27:45.535321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec59847-1943-492c-a465-5976550d4fb3 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:27:45.535585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec59847-1943-492c-a465-5976550d4fb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:27:45.543428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec59847-1943-492c-a465-5976550d4fb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:27:45.544992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec59847-1943-492c-a465-5976550d4fb3 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:27:45.553420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1317/2611] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Aug 23 08:27:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 08:27:55.481714 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:27:55.550216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:28:17.962688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698969ca-4e96-4cdf-a381-1b86ee1e5019 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:17.972838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698969ca-4e96-4cdf-a381-1b86ee1e5019 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:17.976500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ee6bd-4dda-43e6-9ac1-7b7288d51abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:17.977093 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ee6bd-4dda-43e6-9ac1-7b7288d51abf service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:17.977370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ee6bd-4dda-43e6-9ac1-7b7288d51abf service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:17.977642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ee6bd-4dda-43e6-9ac1-7b7288d51abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:17.979451 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-7f4ee6bd-4dda-43e6-9ac1-7b7288d51abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:17.989038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ee6bd-4dda-43e6-9ac1-7b7288d51abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:17.990456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ee6bd-4dda-43e6-9ac1-7b7288d51abf service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:17.998984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1295/2612] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:28:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:28:25.427708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0613f00-0423-4f25-ae75-0f7aea88ee49 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:25.428291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0613f00-0423-4f25-ae75-0f7aea88ee49 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:25.428291 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0613f00-0423-4f25-ae75-0f7aea88ee49 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:25.429617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1318/2613] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:28:25.436476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204f78e-ef11-4b5e-8dcb-e664cd64f634 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:25.437346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204f78e-ef11-4b5e-8dcb-e664cd64f634 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:25.437346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204f78e-ef11-4b5e-8dcb-e664cd64f634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:25.475259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-d204f78e-ef11-4b5e-8dcb-e664cd64f634 None None] MFA Rules not processed for user `d99632164b3b4a1f95736a65da217db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:28:25.476481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d204f78e-ef11-4b5e-8dcb-e664cd64f634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:25.483738 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1296/2614] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:28:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:28:25.496358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0613f00-0423-4f25-ae75-0f7aea88ee49 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:25.507554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0613f00-0423-4f25-ae75-0f7aea88ee49 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:25.511243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9307767-a70e-4d78-9665-ebf820f68860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:25.512277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9307767-a70e-4d78-9665-ebf820f68860 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:25.512277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9307767-a70e-4d78-9665-ebf820f68860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:25.512514 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9307767-a70e-4d78-9665-ebf820f68860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:25.519656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9307767-a70e-4d78-9665-ebf820f68860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:25.521107 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9307767-a70e-4d78-9665-ebf820f68860 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:25.529593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1319/2615] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:28:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:28:26.623784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18222-1cd3-477b-bf8f-fbf1eba25ddc None None] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:26.624260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18222-1cd3-477b-bf8f-fbf1eba25ddc None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:26.624377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d18222-1cd3-477b-bf8f-fbf1eba25ddc None None] PATH_INFO: `/` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:26.625722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1297/2616] 10.222.0.58 () {54 vars in 921 bytes} [Tue Aug 23 08:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 08:28:26.630757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19c79-1eff-49b2-a67d-fa7b204511ba None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:26.631239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19c79-1eff-49b2-a67d-fa7b204511ba None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:26.631356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19c79-1eff-49b2-a67d-fa7b204511ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:26.668974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-66d19c79-1eff-49b2-a67d-fa7b204511ba None None] MFA Rules not processed for user `03cc4435b9b0444b9f278e15398443c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:28:26.670167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-66d19c79-1eff-49b2-a67d-fa7b204511ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:26.679144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1320/2617] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Aug 23 08:28:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:28:26.689528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d18222-1cd3-477b-bf8f-fbf1eba25ddc None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:26.700891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d18222-1cd3-477b-bf8f-fbf1eba25ddc None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:26.704914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed5e0b-7d0a-4c90-ae6e-2e292b0c10e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:26.705691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed5e0b-7d0a-4c90-ae6e-2e292b0c10e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:26.706129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed5e0b-7d0a-4c90-ae6e-2e292b0c10e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:26.706581 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed5e0b-7d0a-4c90-ae6e-2e292b0c10e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:26.717769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed5e0b-7d0a-4c90-ae6e-2e292b0c10e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:26.719208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed5e0b-7d0a-4c90-ae6e-2e292b0c10e9 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:26.730343 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1298/2618] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:28:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:28:26.791248 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d19c79-1eff-49b2-a67d-fa7b204511ba None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:26.792635 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-66d19c79-1eff-49b2-a67d-fa7b204511ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:26.860538 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d19c79-1eff-49b2-a67d-fa7b204511ba None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:26.865808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:26.866407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:26.866689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:26.866987 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:26.868709 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:26.879949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:26.881525 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:26.892386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1321/2619] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:28:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:28:27.303720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed5e0b-7d0a-4c90-ae6e-2e292b0c10e9 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:27.315617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed5e0b-7d0a-4c90-ae6e-2e292b0c10e9 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:27.321196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:27.328833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:27.328833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:27.328833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:27.328833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:27.336533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:27.338171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:27.349085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1299/2620] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:28:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:28:36.888385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=47 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:28:37.344463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=27 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:28:47.179516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:47.180963 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:47.197071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69749c2-ee00-47be-b32e-bed6487931c1 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:47.203470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:47.204159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:47.204499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:47.204835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:47.206584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:47.219164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:47.220744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:47.232942 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1322/2621] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:28:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:28:52.023383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:52.033306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0e336-79e4-4bb1-8467-7bac32eb7f7a service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:52.036920 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:52.037520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:52.037789 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:52.038064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:52.039821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:52.102392 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:52.103854 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:52.114087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1300/2622] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:28:55.753837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:55.755388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:55.771816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f2f51-bf46-4c47-bd48-43361fe5cf4f service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:55.776350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:55.777008 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:55.777289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:55.777589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] PATH_INFO: `/v3/users/ce316aa5c0b54ac99ade6cc4163ee209` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:55.778707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce316aa5c0b54ac99ade6cc4163ee209)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:55.780725 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:56.047056 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce316aa5c0b54ac99ade6cc4163ee209'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.053989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce316aa5c0b54ac99ade6cc4163ee209'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.054645 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce316aa5c0b54ac99ade6cc4163ee209'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.055245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce316aa5c0b54ac99ade6cc4163ee209'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.066836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-48204066-ab30-4c68-b399-08510a9f8a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce316aa5c0b54ac99ade6cc4163ee209'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.170025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1323/2623] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:28:55 2022] DELETE /identity/v3/users/ce316aa5c0b54ac99ade6cc4163ee209 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:28:56.422931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:56.435068 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98c005-4570-4e78-bf8f-1abdcee042e1 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:56.489852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:56.490476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:56.490766 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:56.491059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] PATH_INFO: `/v3/projects/bc3167dc177744959760f39d57937670` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:56.492176 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3167dc177744959760f39d57937670)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:56.494079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:56.633244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3167dc177744959760f39d57937670', 'request_id': 'req-f6e77859-2d36-423b-a0a2-142e102a801f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.638839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3167dc177744959760f39d57937670', 'request_id': 'req-f6e77859-2d36-423b-a0a2-142e102a801f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.645001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3167dc177744959760f39d57937670', 'request_id': 'req-f6e77859-2d36-423b-a0a2-142e102a801f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.758436 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] The token cache is being invalidate because project bc3167dc177744959760f39d57937670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:28:56.758625 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.759260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f6e77859-2d36-423b-a0a2-142e102a801f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:28:56.760773 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1301/2624] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:28:56 2022] DELETE /identity/v3/projects/bc3167dc177744959760f39d57937670 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:28:56.766035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acca8-ba45-4ca6-a974-216d0d84d3de None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:56.766496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acca8-ba45-4ca6-a974-216d0d84d3de None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:56.766614 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856acca8-ba45-4ca6-a974-216d0d84d3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:56.786649 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-856acca8-ba45-4ca6-a974-216d0d84d3de None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:28:56.787887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-856acca8-ba45-4ca6-a974-216d0d84d3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:56.847133 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1324/2625] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:28:56.855071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9644cca-9755-4dcb-bdea-83bdd34412b6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:56.855873 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9644cca-9755-4dcb-bdea-83bdd34412b6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:56.856157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9644cca-9755-4dcb-bdea-83bdd34412b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:56.876230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e9644cca-9755-4dcb-bdea-83bdd34412b6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:28:56.877429 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e9644cca-9755-4dcb-bdea-83bdd34412b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:56.885313 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1302/2626] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:28:56.889547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856acca8-ba45-4ca6-a974-216d0d84d3de None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:56.899234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856acca8-ba45-4ca6-a974-216d0d84d3de None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:56.903811 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:56.904430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:56.904711 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:56.904984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:56.905694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:28:56.906002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:56.907720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:56.919717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1325/2627] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:28:56.923761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9644cca-9755-4dcb-bdea-83bdd34412b6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:56.932950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9644cca-9755-4dcb-bdea-83bdd34412b6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:56.938272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f199060-08e0-4fce-af4b-dbcbc436b03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:56.938893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f199060-08e0-4fce-af4b-dbcbc436b03e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:56.939206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f199060-08e0-4fce-af4b-dbcbc436b03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:56.939479 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f199060-08e0-4fce-af4b-dbcbc436b03e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:56.940092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f199060-08e0-4fce-af4b-dbcbc436b03e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:56.942047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f199060-08e0-4fce-af4b-dbcbc436b03e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:56.958831 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:28:57.073722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1303/2628] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:28:56 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:28:57.078319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:57.088642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccb1cf-0b0f-4a85-aea4-584b6e5d9e2b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:57.133221 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:28:57.145247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04238f4a-64ea-4a08-81c0-49924a783262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:57.145901 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04238f4a-64ea-4a08-81c0-49924a783262 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:57.146177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04238f4a-64ea-4a08-81c0-49924a783262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:57.146445 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04238f4a-64ea-4a08-81c0-49924a783262 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:57.147097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04238f4a-64ea-4a08-81c0-49924a783262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:57.149037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04238f4a-64ea-4a08-81c0-49924a783262 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:57.301292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1326/2629] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:28:57 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:28:57.305690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f199060-08e0-4fce-af4b-dbcbc436b03e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:57.316874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f199060-08e0-4fce-af4b-dbcbc436b03e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:57.321675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587ad13-c83e-4a95-818a-9944341e001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:57.322414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ad13-c83e-4a95-818a-9944341e001a admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:57.322668 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ad13-c83e-4a95-818a-9944341e001a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:57.322966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587ad13-c83e-4a95-818a-9944341e001a admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:57.323661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587ad13-c83e-4a95-818a-9944341e001a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:57.326157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587ad13-c83e-4a95-818a-9944341e001a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:57.336510 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1304/2630] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:28:57.340777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04238f4a-64ea-4a08-81c0-49924a783262 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:57.351111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04238f4a-64ea-4a08-81c0-49924a783262 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:57.355764 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1a5ec-5885-4d2b-a435-6fd041434872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:57.356351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1a5ec-5885-4d2b-a435-6fd041434872 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:57.356627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1a5ec-5885-4d2b-a435-6fd041434872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:57.356894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1a5ec-5885-4d2b-a435-6fd041434872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:57.357524 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1a5ec-5885-4d2b-a435-6fd041434872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:57.359150 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1a5ec-5885-4d2b-a435-6fd041434872 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:57.369454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1327/2631] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:28:57.373974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587ad13-c83e-4a95-818a-9944341e001a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:57.383832 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587ad13-c83e-4a95-818a-9944341e001a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:57.389036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:57.389664 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:57.389938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:57.390206 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] PATH_INFO: `/v3/projects/1367659f7d6a404393adca1c5d713c99/users/d780debf950447b983dc94edfc681a68/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:57.404597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1367659f7d6a404393adca1c5d713c99, user_id=d780debf950447b983dc94edfc681a68, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:57.407273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:57.524369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1305/2632] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:28:57 2022] PUT /identity/v3/projects/1367659f7d6a404393adca1c5d713c99/users/d780debf950447b983dc94edfc681a68/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:28:57.530052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114350e-8f5e-46d4-81a7-c3a0c100f73f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:57.531308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114350e-8f5e-46d4-81a7-c3a0c100f73f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:57.531592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114350e-8f5e-46d4-81a7-c3a0c100f73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:57.551958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7114350e-8f5e-46d4-81a7-c3a0c100f73f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:28:57.553149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7114350e-8f5e-46d4-81a7-c3a0c100f73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:57.613507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1328/2633] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:28:57.623821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:28:57.625586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:28:57.682103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d222137-5322-44b2-a637-1e5d75bfaea7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:28:57.686357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:28:57.686976 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:28:57.687262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:28:57.687536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:28:57.696813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:28:57.698243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:28:57.708594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1306/2634] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:29:02.944135 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c5c9d-3c8a-4015-a59e-69a3207c6335 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:29:02.944707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c5c9d-3c8a-4015-a59e-69a3207c6335 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:29:02.944707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c5c9d-3c8a-4015-a59e-69a3207c6335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:29:02.975001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e87c5c9d-3c8a-4015-a59e-69a3207c6335 None None] MFA Rules not processed for user `d780debf950447b983dc94edfc681a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:29:02.976243 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e87c5c9d-3c8a-4015-a59e-69a3207c6335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:29:03.041973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1329/2635] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:29:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:29:03.056884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:29:03.058724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:29:03.134932 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f590a-875c-44ec-ab7d-ac56b93ef541 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:29:03.140650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:29:03.141329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:29:03.141611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:29:03.141885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:29:03.149662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:29:03.151314 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:29:03.160293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1307/2636] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:29:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:29:05.401211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b2627-3c7d-4623-9a69-f581e0bbb5c6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:29:05.401211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b2627-3c7d-4623-9a69-f581e0bbb5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:29:05.401211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b2627-3c7d-4623-9a69-f581e0bbb5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:29:05.423996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-482b2627-3c7d-4623-9a69-f581e0bbb5c6 None None] MFA Rules not processed for user `d780debf950447b983dc94edfc681a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:29:05.425183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-482b2627-3c7d-4623-9a69-f581e0bbb5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:29:05.441443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1330/2637] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:29:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:29:05.775230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:29:05.777644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:29:05.790376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501aeb54-6735-4074-b716-7bbb7c5bf4a7 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:29:05.794721 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:29:05.795566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:29:05.795983 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:29:05.796268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:29:05.798056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:29:05.808520 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:29:05.809975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:29:05.818691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1308/2638] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:29:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:29:15.418327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:29:15.822147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:30:24.301100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b2627-3c7d-4623-9a69-f581e0bbb5c6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:24.311772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b2627-3c7d-4623-9a69-f581e0bbb5c6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:24.315537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:24.316134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:24.316411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:24.316683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:24.318444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:24.377384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:24.378858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:24.387680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1331/2639] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:30:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:30:30.847420 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:30.856847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba611a3-e75e-486e-b3e3-0f54c5fb10df service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:30.860822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:30.861450 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:30.861730 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:30.862028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:30.863883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:30.876757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:30.878309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:30.888565 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1309/2640] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:30:34.383864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=28 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:30:38.922278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:38.924107 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:39.012089 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccbe63-46c3-45e6-9959-c385737f544e service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:39.018050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:39.018607 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:39.018894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:39.019235 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:39.020982 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:39.032744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:39.034713 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:39.045876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1332/2641] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:30:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:30:39.059205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:39.064364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:39.077268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d74320-7dfa-4d1c-8e15-7f492fac159b service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:39.082582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:39.083690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:39.084132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:39.084917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] PATH_INFO: `/v3/users/ef9753669de440489453104bbe61b449` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:39.086406 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9753669de440489453104bbe61b449)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:39.088386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:39.262829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9753669de440489453104bbe61b449'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.263516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9753669de440489453104bbe61b449'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.271423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9753669de440489453104bbe61b449'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.272102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9753669de440489453104bbe61b449'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.346205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9753669de440489453104bbe61b449'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.364278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1310/2642] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:30:39 2022] DELETE /identity/v3/users/ef9753669de440489453104bbe61b449 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:30:39.634572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:39.653093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21377131-8541-4a89-91fd-99dcb7bf8a1f service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:39.721094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:39.721727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:39.722000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:39.722266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] PATH_INFO: `/v3/projects/f80bce57227744ac9be2fda344a86e39` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:39.723422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80bce57227744ac9be2fda344a86e39)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:39.725332 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:39.844621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80bce57227744ac9be2fda344a86e39', 'request_id': 'req-c22b8b4f-3647-4a68-85e9-0c2f75949f60'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.912022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80bce57227744ac9be2fda344a86e39', 'request_id': 'req-c22b8b4f-3647-4a68-85e9-0c2f75949f60'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.917977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80bce57227744ac9be2fda344a86e39', 'request_id': 'req-c22b8b4f-3647-4a68-85e9-0c2f75949f60'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.929172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] The token cache is being invalidate because project f80bce57227744ac9be2fda344a86e39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:30:39.929739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.930066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:39.931574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1333/2643] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:30:39 2022] DELETE /identity/v3/projects/f80bce57227744ac9be2fda344a86e39 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:30:39.935650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:39.937291 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:39.950583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ea434-6888-4552-848b-9baa5ea7f523 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:40.007045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:40.007673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:40.007978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:40.008252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] PATH_INFO: `/v3/users/52cb54620c1c41a8ad17e15f2b3c1da7` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:40.009433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52cb54620c1c41a8ad17e15f2b3c1da7)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:40.011433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:40.254399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52cb54620c1c41a8ad17e15f2b3c1da7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.255056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52cb54620c1c41a8ad17e15f2b3c1da7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.261925 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52cb54620c1c41a8ad17e15f2b3c1da7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.262548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52cb54620c1c41a8ad17e15f2b3c1da7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.266470 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:30:40.329155 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f02096b1-9f57-46e9-9fc0-80e0b29a0b40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52cb54620c1c41a8ad17e15f2b3c1da7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.347226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1311/2644] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:30:39 2022] DELETE /identity/v3/users/52cb54620c1c41a8ad17e15f2b3c1da7 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:30:40.598824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:40.600320 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:40.614112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b8b4f-3647-4a68-85e9-0c2f75949f60 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:40.633894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:30:40.674694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:40.675388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:40.675674 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:40.675948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] PATH_INFO: `/v3/projects/b358a54339494b99bf299ed88ac0e06c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:40.677085 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b358a54339494b99bf299ed88ac0e06c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:40.679108 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:40.794608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b358a54339494b99bf299ed88ac0e06c', 'request_id': 'req-e72c60b4-ce16-474a-9cf8-9be3c77294af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.878197 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b358a54339494b99bf299ed88ac0e06c', 'request_id': 'req-e72c60b4-ce16-474a-9cf8-9be3c77294af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.884915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b358a54339494b99bf299ed88ac0e06c', 'request_id': 'req-e72c60b4-ce16-474a-9cf8-9be3c77294af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.896397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] The token cache is being invalidate because project b358a54339494b99bf299ed88ac0e06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:30:40.896687 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.897331 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e72c60b4-ce16-474a-9cf8-9be3c77294af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:30:40.898825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1334/2645] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:30:40 2022] DELETE /identity/v3/projects/b358a54339494b99bf299ed88ac0e06c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:30:40.903930 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0fb8a-cfa5-4632-9f10-052a6a3b3556 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:40.904234 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0fb8a-cfa5-4632-9f10-052a6a3b3556 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:40.904571 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0fb8a-cfa5-4632-9f10-052a6a3b3556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:40.925815 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-2cc0fb8a-cfa5-4632-9f10-052a6a3b3556 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:30:40.927079 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-2cc0fb8a-cfa5-4632-9f10-052a6a3b3556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:40.988849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1312/2646] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:30:40.996786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25dde9-aec2-4032-8cee-54946634a2e1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:40.997252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25dde9-aec2-4032-8cee-54946634a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:40.997369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25dde9-aec2-4032-8cee-54946634a2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.016501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cc25dde9-aec2-4032-8cee-54946634a2e1 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:30:41.017731 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cc25dde9-aec2-4032-8cee-54946634a2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:41.025518 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1335/2647] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:30:41.029705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0fb8a-cfa5-4632-9f10-052a6a3b3556 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:41.039912 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0fb8a-cfa5-4632-9f10-052a6a3b3556 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:41.044462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:41.045050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:41.045334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:41.045609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.046254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:30:41.046558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:41.048181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:41.060060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1313/2648] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:30:41.064113 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25dde9-aec2-4032-8cee-54946634a2e1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:41.073339 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25dde9-aec2-4032-8cee-54946634a2e1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:41.078001 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22597cc6-9671-4a92-995f-3c906480c19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:41.078579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22597cc6-9671-4a92-995f-3c906480c19a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:41.078855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22597cc6-9671-4a92-995f-3c906480c19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:41.079166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22597cc6-9671-4a92-995f-3c906480c19a admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.079829 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22597cc6-9671-4a92-995f-3c906480c19a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:41.081710 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22597cc6-9671-4a92-995f-3c906480c19a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:41.165822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1336/2649] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:30:41 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:30:41.170094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:41.180523 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db3035-463e-4c35-866a-be8e80d6e49e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:41.237147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375d4ea-469a-49b7-ac9d-4f3f7dd62f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:41.237744 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375d4ea-469a-49b7-ac9d-4f3f7dd62f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:41.238027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375d4ea-469a-49b7-ac9d-4f3f7dd62f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:41.238300 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375d4ea-469a-49b7-ac9d-4f3f7dd62f14 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.238986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375d4ea-469a-49b7-ac9d-4f3f7dd62f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:41.240877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375d4ea-469a-49b7-ac9d-4f3f7dd62f14 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:41.306605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1314/2650] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:30:41 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:30:41.319990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22597cc6-9671-4a92-995f-3c906480c19a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:41.331436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22597cc6-9671-4a92-995f-3c906480c19a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:41.336118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac4cfe-a9c0-475f-a29e-e484ffebe057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:41.336750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4cfe-a9c0-475f-a29e-e484ffebe057 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:41.337048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4cfe-a9c0-475f-a29e-e484ffebe057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:41.337323 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac4cfe-a9c0-475f-a29e-e484ffebe057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.337954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac4cfe-a9c0-475f-a29e-e484ffebe057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:41.339611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac4cfe-a9c0-475f-a29e-e484ffebe057 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:41.350030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1337/2651] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:30:41.353887 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375d4ea-469a-49b7-ac9d-4f3f7dd62f14 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:41.364169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375d4ea-469a-49b7-ac9d-4f3f7dd62f14 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:41.368900 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b2604-ab0e-48c9-8056-f96f7df5b8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:41.369488 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b2604-ab0e-48c9-8056-f96f7df5b8ff admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:41.369771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b2604-ab0e-48c9-8056-f96f7df5b8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:41.370047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b2604-ab0e-48c9-8056-f96f7df5b8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.370688 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b2604-ab0e-48c9-8056-f96f7df5b8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:41.372322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b2604-ab0e-48c9-8056-f96f7df5b8ff admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:41.382889 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1315/2652] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:30:41.386874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac4cfe-a9c0-475f-a29e-e484ffebe057 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:41.397274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac4cfe-a9c0-475f-a29e-e484ffebe057 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:41.401855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:41.402480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:41.402753 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:41.403049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] PATH_INFO: `/v3/projects/7625c9efcc924680a5efef3fd71d0272/users/db8b5a8edcc34d32a3e9fad54d33cbbc/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.417909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7625c9efcc924680a5efef3fd71d0272, user_id=db8b5a8edcc34d32a3e9fad54d33cbbc, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:41.420585 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:41.500885 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1338/2653] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:30:41 2022] PUT /identity/v3/projects/7625c9efcc924680a5efef3fd71d0272/users/db8b5a8edcc34d32a3e9fad54d33cbbc/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:30:41.506385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8358bc5-3058-4d57-bdc5-44e463b0f4ad None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:41.506673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8358bc5-3058-4d57-bdc5-44e463b0f4ad None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:41.506979 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8358bc5-3058-4d57-bdc5-44e463b0f4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.525950 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-e8358bc5-3058-4d57-bdc5-44e463b0f4ad None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:30:41.527174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-e8358bc5-3058-4d57-bdc5-44e463b0f4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:41.567073 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:30:41.586076 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1316/2654] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:30:41.596041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:41.597698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:41.649701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:30:41.655356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93da11-99b7-4eaa-ad00-65a74296055a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:41.659220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a0c1b-4845-49d5-a299-88a4d5f48fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:41.659821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a0c1b-4845-49d5-a299-88a4d5f48fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:41.660096 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a0c1b-4845-49d5-a299-88a4d5f48fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:41.660365 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a0c1b-4845-49d5-a299-88a4d5f48fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:41.669654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a0c1b-4845-49d5-a299-88a4d5f48fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:41.671158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a0c1b-4845-49d5-a299-88a4d5f48fa6 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:41.680608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1339/2655] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:30:46.870910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437389e-d01e-4f78-94a4-70a3499c7c22 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:46.871821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437389e-d01e-4f78-94a4-70a3499c7c22 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:46.872105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437389e-d01e-4f78-94a4-70a3499c7c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:46.902055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1437389e-d01e-4f78-94a4-70a3499c7c22 None None] MFA Rules not processed for user `db8b5a8edcc34d32a3e9fad54d33cbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:30:46.903309 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1437389e-d01e-4f78-94a4-70a3499c7c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:46.973445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1317/2656] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:30:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:30:46.978045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a0c1b-4845-49d5-a299-88a4d5f48fa6 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:46.988854 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a0c1b-4845-49d5-a299-88a4d5f48fa6 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:46.993636 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09339e-cb32-4b9c-a68e-f8e0c353d5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:46.994241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09339e-cb32-4b9c-a68e-f8e0c353d5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:46.994516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09339e-cb32-4b9c-a68e-f8e0c353d5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:46.994783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09339e-cb32-4b9c-a68e-f8e0c353d5a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:46.995527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09339e-cb32-4b9c-a68e-f8e0c353d5a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:46.997425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09339e-cb32-4b9c-a68e-f8e0c353d5a0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:47.080210 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1340/2657] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:30:46 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:30:47.084379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437389e-d01e-4f78-94a4-70a3499c7c22 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:47.096238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437389e-d01e-4f78-94a4-70a3499c7c22 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:47.163336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869c741-b978-47fe-b448-764d6c0c3517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:47.163963 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869c741-b978-47fe-b448-764d6c0c3517 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:47.164245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869c741-b978-47fe-b448-764d6c0c3517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:47.164695 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869c741-b978-47fe-b448-764d6c0c3517 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:47.165349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869c741-b978-47fe-b448-764d6c0c3517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:47.167503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869c741-b978-47fe-b448-764d6c0c3517 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:47.301470 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1318/2658] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:30:47 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:30:47.305426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09339e-cb32-4b9c-a68e-f8e0c353d5a0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:47.315948 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09339e-cb32-4b9c-a68e-f8e0c353d5a0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:47.320564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033202b-a679-4bcb-892c-70a394bf7f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:47.321452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033202b-a679-4bcb-892c-70a394bf7f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:47.321761 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033202b-a679-4bcb-892c-70a394bf7f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:47.322034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033202b-a679-4bcb-892c-70a394bf7f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:47.322698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033202b-a679-4bcb-892c-70a394bf7f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:47.324341 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033202b-a679-4bcb-892c-70a394bf7f16 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:47.337086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1341/2659] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:30:47.341361 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869c741-b978-47fe-b448-764d6c0c3517 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:47.355334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869c741-b978-47fe-b448-764d6c0c3517 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:47.360975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8336d-efee-4778-bd42-3217e62270c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:47.361778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8336d-efee-4778-bd42-3217e62270c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:47.362154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8336d-efee-4778-bd42-3217e62270c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:47.362497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8336d-efee-4778-bd42-3217e62270c3 admin admin] PATH_INFO: `/v3/projects/21c42631bb974a16a633bc24b885dd41/users/c113f1533907487e81ddb547c31b26dd/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:47.379148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8336d-efee-4778-bd42-3217e62270c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c42631bb974a16a633bc24b885dd41, user_id=c113f1533907487e81ddb547c31b26dd, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:47.381846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8336d-efee-4778-bd42-3217e62270c3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:47.450514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1319/2660] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:30:47 2022] PUT /identity/v3/projects/21c42631bb974a16a633bc24b885dd41/users/c113f1533907487e81ddb547c31b26dd/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:30:47.455116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033202b-a679-4bcb-892c-70a394bf7f16 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:47.465766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033202b-a679-4bcb-892c-70a394bf7f16 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:47.524144 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f441e5-4106-45ac-8a68-e971e4eb4f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:47.524754 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f441e5-4106-45ac-8a68-e971e4eb4f80 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:47.525035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f441e5-4106-45ac-8a68-e971e4eb4f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:47.525306 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f441e5-4106-45ac-8a68-e971e4eb4f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:47.526038 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f441e5-4106-45ac-8a68-e971e4eb4f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:47.527573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f441e5-4106-45ac-8a68-e971e4eb4f80 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:47.538382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1342/2661] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:30:47.542697 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8336d-efee-4778-bd42-3217e62270c3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:47.554461 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8336d-efee-4778-bd42-3217e62270c3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:47.559647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:47.560345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:47.560788 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:47.561069 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] PATH_INFO: `/v3/projects/21c42631bb974a16a633bc24b885dd41/users/c113f1533907487e81ddb547c31b26dd/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:47.563970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c42631bb974a16a633bc24b885dd41, user_id=c113f1533907487e81ddb547c31b26dd, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:47.570281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:47.580476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1320/2662] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:30:47 2022] PUT /identity/v3/projects/21c42631bb974a16a633bc24b885dd41/users/c113f1533907487e81ddb547c31b26dd/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:30:50.096606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502348-60fa-485d-b19d-3b9fae65433d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:50.097143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502348-60fa-485d-b19d-3b9fae65433d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:50.097143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502348-60fa-485d-b19d-3b9fae65433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:50.122632 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ac502348-60fa-485d-b19d-3b9fae65433d None None] MFA Rules not processed for user `d780debf950447b983dc94edfc681a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:30:50.126491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ac502348-60fa-485d-b19d-3b9fae65433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:50.212873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1343/2663] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:30:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:30:50.603941 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:50.606345 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:50.655775 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:30:50.774265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fac25f-86e3-4eab-a08f-9fddd5e0618a admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:50.778724 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb3e42-df0c-49f2-9882-89ba9cae01bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:50.790794 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb3e42-df0c-49f2-9882-89ba9cae01bb service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:50.791685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb3e42-df0c-49f2-9882-89ba9cae01bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:50.791685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb3e42-df0c-49f2-9882-89ba9cae01bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:50.800279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-78fb3e42-df0c-49f2-9882-89ba9cae01bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:50.915331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb3e42-df0c-49f2-9882-89ba9cae01bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:50.917004 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb3e42-df0c-49f2-9882-89ba9cae01bb service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:50.930438 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1321/2664] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:30:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:30:50.941103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502348-60fa-485d-b19d-3b9fae65433d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:50.941103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ac502348-60fa-485d-b19d-3b9fae65433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:51.074178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502348-60fa-485d-b19d-3b9fae65433d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:51.078542 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:51.079466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:51.079793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:51.080134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:51.082832 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:51.102657 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:51.105073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:51.111549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:30:51.131167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1344/2665] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:30:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:30:54.461262 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a741d73-f8cd-4716-909b-5ffc3ba636eb None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:54.462196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a741d73-f8cd-4716-909b-5ffc3ba636eb None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:54.467600 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a741d73-f8cd-4716-909b-5ffc3ba636eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:54.520171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-3a741d73-f8cd-4716-909b-5ffc3ba636eb None None] MFA Rules not processed for user `c113f1533907487e81ddb547c31b26dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:30:54.523659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3a741d73-f8cd-4716-909b-5ffc3ba636eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:54.673432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1322/2666] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:30:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:30:54.680448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:54.682016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:54.800644 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9ca22-ff24-41be-8f33-a9d3de36dd90 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:54.806830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0a3a0-d903-40a3-be4d-1aeaff0b9a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:54.807575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a3a0-d903-40a3-be4d-1aeaff0b9a5b service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:54.807862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a3a0-d903-40a3-be4d-1aeaff0b9a5b service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:54.808137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0a3a0-d903-40a3-be4d-1aeaff0b9a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:54.875464 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0a3a0-d903-40a3-be4d-1aeaff0b9a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:54.880935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0a3a0-d903-40a3-be4d-1aeaff0b9a5b service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:54.893575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1345/2667] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:30:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:30:56.475252 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a741d73-f8cd-4716-909b-5ffc3ba636eb None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:30:56.477078 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3a741d73-f8cd-4716-909b-5ffc3ba636eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:56.500167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a741d73-f8cd-4716-909b-5ffc3ba636eb None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:30:56.506878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:30:56.507550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:30:56.507984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:30:56.508373 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:30:56.511127 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:30:56.527422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:30:56.529292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:30:56.573217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1323/2668] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:30:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:31:00.874786 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4780c4-74be-4416-9dd5-5bfa70236751 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:31:00.875411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4780c4-74be-4416-9dd5-5bfa70236751 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:31:00.875411 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4780c4-74be-4416-9dd5-5bfa70236751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:31:00.894481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7e4780c4-74be-4416-9dd5-5bfa70236751 None None] MFA Rules not processed for user `6960cb25d98c4b02a80900d97ab34a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:31:00.895720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7e4780c4-74be-4416-9dd5-5bfa70236751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:31:00.902584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1346/2669] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:31:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:31:06.537898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=85 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:31:10.888803 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=59 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:31:16.154567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:31:16.156016 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:31:16.225269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed0ff8-523a-4f93-b28b-90c5e50cf9a4 service nova] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:31:16.230250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:31:16.230845 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:31:16.231148 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:31:16.231420 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:31:16.233119 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:31:16.244897 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:31:16.246476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:31:16.256124 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1324/2670] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:31:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:31:20.315646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bcd81-eed8-4dc8-a626-22889afd6388 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:31:20.316239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bcd81-eed8-4dc8-a626-22889afd6388 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:31:20.316239 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bcd81-eed8-4dc8-a626-22889afd6388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:31:20.336472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7e5bcd81-eed8-4dc8-a626-22889afd6388 None None] MFA Rules not processed for user `db8b5a8edcc34d32a3e9fad54d33cbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:31:20.337705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7e5bcd81-eed8-4dc8-a626-22889afd6388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:31:20.344925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1347/2671] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:31:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:31:26.252036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=32 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:31:30.330415 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:31:41.321261 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:31:41.330680 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbd0a5-cd33-4163-89d0-d76b3746f80c service placement] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:31:41.334465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42dcaf-9789-49fe-8e3d-92eadcdd09be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:31:41.335164 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42dcaf-9789-49fe-8e3d-92eadcdd09be service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:31:41.335448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42dcaf-9789-49fe-8e3d-92eadcdd09be service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:31:41.336213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42dcaf-9789-49fe-8e3d-92eadcdd09be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:31:41.345060 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42dcaf-9789-49fe-8e3d-92eadcdd09be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:31:41.346511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42dcaf-9789-49fe-8e3d-92eadcdd09be service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:31:41.356652 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1325/2672] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:31:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:31:51.351868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:32:24.095587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bcd81-eed8-4dc8-a626-22889afd6388 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:32:24.097147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7e5bcd81-eed8-4dc8-a626-22889afd6388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:32:24.109591 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bcd81-eed8-4dc8-a626-22889afd6388 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:32:24.113192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f79619-8304-40b5-83b9-73cdab4e97ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:32:24.113780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79619-8304-40b5-83b9-73cdab4e97ae service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:32:24.114052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79619-8304-40b5-83b9-73cdab4e97ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:32:24.114315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79619-8304-40b5-83b9-73cdab4e97ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:32:24.116091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-34f79619-8304-40b5-83b9-73cdab4e97ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:32:24.128934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f79619-8304-40b5-83b9-73cdab4e97ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:32:24.130343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f79619-8304-40b5-83b9-73cdab4e97ae service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:32:24.141080 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1348/2673] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:32:25.701057 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42dcaf-9789-49fe-8e3d-92eadcdd09be service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:32:25.709749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42dcaf-9789-49fe-8e3d-92eadcdd09be service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:32:25.713412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:32:25.713998 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:32:25.714278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:32:25.714551 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:32:25.716327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:32:25.729629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:32:25.731097 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:32:25.741988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1326/2674] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:32:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:32:34.136314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:32:35.737172 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:32:40.715724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8248efd-33d9-4fce-b933-dd9e55e9e1e9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:32:40.716245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8248efd-33d9-4fce-b933-dd9e55e9e1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:32:40.716245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8248efd-33d9-4fce-b933-dd9e55e9e1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:32:40.735755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f8248efd-33d9-4fce-b933-dd9e55e9e1e9 None None] MFA Rules not processed for user `d780debf950447b983dc94edfc681a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:32:40.736945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f8248efd-33d9-4fce-b933-dd9e55e9e1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:32:40.745020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1349/2675] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:32:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:32:50.731023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:11.976582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:11.985973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7339397-b168-4613-9354-79fed8042603 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:11.989722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:11.990313 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:11.990584 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:11.990858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:11.992610 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:12.054630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:12.056041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:12.065541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1327/2676] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:15.636112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8248efd-33d9-4fce-b933-dd9e55e9e1e9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:15.637564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f8248efd-33d9-4fce-b933-dd9e55e9e1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:15.653806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8248efd-33d9-4fce-b933-dd9e55e9e1e9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:15.658378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:15.659037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:15.659322 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:15.659616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] PATH_INFO: `/v3/users/6960cb25d98c4b02a80900d97ab34a5a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:15.660741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6960cb25d98c4b02a80900d97ab34a5a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:15.662820 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:15.838972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6960cb25d98c4b02a80900d97ab34a5a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:15.846778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6960cb25d98c4b02a80900d97ab34a5a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:15.846778 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6960cb25d98c4b02a80900d97ab34a5a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:15.847494 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6960cb25d98c4b02a80900d97ab34a5a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:15.860481 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83ddbe87-e3e4-4529-bb80-ea7e2f23194c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6960cb25d98c4b02a80900d97ab34a5a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:15.948903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1350/2677] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:33:15 2022] DELETE /identity/v3/users/6960cb25d98c4b02a80900d97ab34a5a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:16.191196 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:16.204805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219574c-2142-411d-ae53-5d576b030a68 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:16.263847 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:16.264487 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:16.264769 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:16.265043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] PATH_INFO: `/v3/projects/6c920ebfb7b046e08503e02473294eab` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:16.266200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c920ebfb7b046e08503e02473294eab)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:16.268157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:16.374336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c920ebfb7b046e08503e02473294eab', 'request_id': 'req-c138ad48-15a8-48a4-a582-9ca3a551b729'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:16.380367 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c920ebfb7b046e08503e02473294eab', 'request_id': 'req-c138ad48-15a8-48a4-a582-9ca3a551b729'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:16.388128 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c920ebfb7b046e08503e02473294eab', 'request_id': 'req-c138ad48-15a8-48a4-a582-9ca3a551b729'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:16.445914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] The token cache is being invalidate because project 6c920ebfb7b046e08503e02473294eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:33:16.446205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:16.447134 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c138ad48-15a8-48a4-a582-9ca3a551b729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:16.448720 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1328/2678] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:33:16 2022] DELETE /identity/v3/projects/6c920ebfb7b046e08503e02473294eab => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:16.456014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c567dc0-752c-4bdb-95c4-a79a85d9e7ff None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:16.456399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c567dc0-752c-4bdb-95c4-a79a85d9e7ff None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:16.456723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c567dc0-752c-4bdb-95c4-a79a85d9e7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:16.477259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6c567dc0-752c-4bdb-95c4-a79a85d9e7ff None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:16.478453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6c567dc0-752c-4bdb-95c4-a79a85d9e7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:16.536748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1351/2679] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:16.544764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed22d9-3e1b-405f-b185-f183858c7ce6 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:16.545053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed22d9-3e1b-405f-b185-f183858c7ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:16.545358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed22d9-3e1b-405f-b185-f183858c7ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:16.565445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-53ed22d9-3e1b-405f-b185-f183858c7ce6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:16.566597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-53ed22d9-3e1b-405f-b185-f183858c7ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:16.574823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1329/2680] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:16.579092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c567dc0-752c-4bdb-95c4-a79a85d9e7ff None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:16.588679 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c567dc0-752c-4bdb-95c4-a79a85d9e7ff None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:16.593161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:16.593745 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:16.594018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:16.594286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:16.595570 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:33:16.595872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:16.597451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:16.608779 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1352/2681] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:33:16.613482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed22d9-3e1b-405f-b185-f183858c7ce6 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:16.622833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed22d9-3e1b-405f-b185-f183858c7ce6 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:16.627564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd4038-0adc-4570-9148-7f3c8c96dc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:16.628149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd4038-0adc-4570-9148-7f3c8c96dc21 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:16.628430 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd4038-0adc-4570-9148-7f3c8c96dc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:16.628705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd4038-0adc-4570-9148-7f3c8c96dc21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:16.629338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd4038-0adc-4570-9148-7f3c8c96dc21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:16.631272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd4038-0adc-4570-9148-7f3c8c96dc21 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:16.647116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:16.739482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1330/2682] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:33:16 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:16.743658 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:16.753845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bfdf2-5c90-4c3a-8a96-adf875344eb5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:16.815583 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c0eac-4104-41ab-8e8a-43863efcb00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:16.816119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c0eac-4104-41ab-8e8a-43863efcb00d admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:16.816397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c0eac-4104-41ab-8e8a-43863efcb00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:16.816669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c0eac-4104-41ab-8e8a-43863efcb00d admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:16.817295 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c0eac-4104-41ab-8e8a-43863efcb00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:16.819242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c0eac-4104-41ab-8e8a-43863efcb00d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:16.827313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:16.937772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1353/2683] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:33:16 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:16.941958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd4038-0adc-4570-9148-7f3c8c96dc21 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:16.952611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd4038-0adc-4570-9148-7f3c8c96dc21 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:16.957934 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550fd7f-4b94-4299-874e-a9ac82ece15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:16.958527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550fd7f-4b94-4299-874e-a9ac82ece15c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:16.958839 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550fd7f-4b94-4299-874e-a9ac82ece15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:16.959157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550fd7f-4b94-4299-874e-a9ac82ece15c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:16.959834 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550fd7f-4b94-4299-874e-a9ac82ece15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:16.961449 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550fd7f-4b94-4299-874e-a9ac82ece15c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:16.972391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1331/2684] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:16.976472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c0eac-4104-41ab-8e8a-43863efcb00d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:16.986736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c0eac-4104-41ab-8e8a-43863efcb00d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:16.991538 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d9006-241e-4ade-ab8b-003867ca1f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:16.992123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d9006-241e-4ade-ab8b-003867ca1f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:16.992450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d9006-241e-4ade-ab8b-003867ca1f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:16.992723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d9006-241e-4ade-ab8b-003867ca1f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:16.993353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d9006-241e-4ade-ab8b-003867ca1f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:16.994989 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d9006-241e-4ade-ab8b-003867ca1f7e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:17.006051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1354/2685] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:17.010527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550fd7f-4b94-4299-874e-a9ac82ece15c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:17.020975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550fd7f-4b94-4299-874e-a9ac82ece15c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:17.025486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e100c-5f1d-471b-a323-95b19a8ce794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:17.026117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e100c-5f1d-471b-a323-95b19a8ce794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.026417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e100c-5f1d-471b-a323-95b19a8ce794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.026682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e100c-5f1d-471b-a323-95b19a8ce794 admin admin] PATH_INFO: `/v3/projects/c7753568052648008578e422ef01b8a5/users/18ef4c2372c441c8b0c1bebc3cbb0291/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.040783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e100c-5f1d-471b-a323-95b19a8ce794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7753568052648008578e422ef01b8a5, user_id=18ef4c2372c441c8b0c1bebc3cbb0291, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:17.043491 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e100c-5f1d-471b-a323-95b19a8ce794 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:17.108578 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1332/2686] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:17 2022] PUT /identity/v3/projects/c7753568052648008578e422ef01b8a5/users/18ef4c2372c441c8b0c1bebc3cbb0291/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:17.119782 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d8fa-72f4-49a6-a702-73fc7200646f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.120101 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d8fa-72f4-49a6-a702-73fc7200646f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.120377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d8fa-72f4-49a6-a702-73fc7200646f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.148592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2db9d8fa-72f4-49a6-a702-73fc7200646f None None] MFA Rules not processed for user `18ef4c2372c441c8b0c1bebc3cbb0291`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:17.149890 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2db9d8fa-72f4-49a6-a702-73fc7200646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:17.213616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1355/2687] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:17.218219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e100c-5f1d-471b-a323-95b19a8ce794 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:17.282408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e100c-5f1d-471b-a323-95b19a8ce794 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:17.287281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00591f-662b-4fbc-8862-1db51da71d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:17.287881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00591f-662b-4fbc-8862-1db51da71d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.288162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00591f-662b-4fbc-8862-1db51da71d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.288550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00591f-662b-4fbc-8862-1db51da71d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.289225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00591f-662b-4fbc-8862-1db51da71d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:17.291150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00591f-662b-4fbc-8862-1db51da71d46 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:17.397030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1333/2688] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:33:17 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:17.401581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9d8fa-72f4-49a6-a702-73fc7200646f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:17.412665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9d8fa-72f4-49a6-a702-73fc7200646f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:17.470918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49068a69-e8e2-4b1f-be93-3d0a27bb1c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:17.471546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49068a69-e8e2-4b1f-be93-3d0a27bb1c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.471825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49068a69-e8e2-4b1f-be93-3d0a27bb1c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.472097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49068a69-e8e2-4b1f-be93-3d0a27bb1c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.472728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49068a69-e8e2-4b1f-be93-3d0a27bb1c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:17.474640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49068a69-e8e2-4b1f-be93-3d0a27bb1c3f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:17.554370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1356/2689] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:33:17 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:17.558622 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00591f-662b-4fbc-8862-1db51da71d46 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:17.569644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00591f-662b-4fbc-8862-1db51da71d46 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:17.574597 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320448a-2b86-473c-9d3c-b2cdf7036be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:17.575213 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320448a-2b86-473c-9d3c-b2cdf7036be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.575489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320448a-2b86-473c-9d3c-b2cdf7036be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.575758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320448a-2b86-473c-9d3c-b2cdf7036be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.576390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320448a-2b86-473c-9d3c-b2cdf7036be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:17.578045 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320448a-2b86-473c-9d3c-b2cdf7036be0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:17.588588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1334/2690] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:17.592676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49068a69-e8e2-4b1f-be93-3d0a27bb1c3f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:17.602661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49068a69-e8e2-4b1f-be93-3d0a27bb1c3f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:17.607449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70258092-9a7a-49e6-b423-db153708d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:17.615154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70258092-9a7a-49e6-b423-db153708d1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.615154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70258092-9a7a-49e6-b423-db153708d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.615154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70258092-9a7a-49e6-b423-db153708d1dd admin admin] PATH_INFO: `/v3/projects/ba69bf6480044db696db7540d51083a4/users/d5d2e13a65a04bde9f2e9aa95c421407/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.623900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70258092-9a7a-49e6-b423-db153708d1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba69bf6480044db696db7540d51083a4, user_id=d5d2e13a65a04bde9f2e9aa95c421407, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:17.627262 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70258092-9a7a-49e6-b423-db153708d1dd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:17.708529 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1357/2691] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:17 2022] PUT /identity/v3/projects/ba69bf6480044db696db7540d51083a4/users/d5d2e13a65a04bde9f2e9aa95c421407/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:33:17.712513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320448a-2b86-473c-9d3c-b2cdf7036be0 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:17.723350 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320448a-2b86-473c-9d3c-b2cdf7036be0 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:17.789432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5e4eb-a4fa-4ceb-b975-6d5d33a0d61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:17.790067 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5e4eb-a4fa-4ceb-b975-6d5d33a0d61c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.790464 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5e4eb-a4fa-4ceb-b975-6d5d33a0d61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.790812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5e4eb-a4fa-4ceb-b975-6d5d33a0d61c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.791504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5e4eb-a4fa-4ceb-b975-6d5d33a0d61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:17.793135 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5e4eb-a4fa-4ceb-b975-6d5d33a0d61c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:17.804156 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1335/2692] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:17.808534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70258092-9a7a-49e6-b423-db153708d1dd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:17.819165 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70258092-9a7a-49e6-b423-db153708d1dd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:17.824074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:17.824715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.824991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.825261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] PATH_INFO: `/v3/projects/ba69bf6480044db696db7540d51083a4/users/d5d2e13a65a04bde9f2e9aa95c421407/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.827721 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba69bf6480044db696db7540d51083a4, user_id=d5d2e13a65a04bde9f2e9aa95c421407, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:17.830422 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:17.842090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1358/2693] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:17 2022] PUT /identity/v3/projects/ba69bf6480044db696db7540d51083a4/users/d5d2e13a65a04bde9f2e9aa95c421407/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:17.853228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecb142-7e7b-4e6f-9be4-0c76fdf82b81 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:17.853516 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecb142-7e7b-4e6f-9be4-0c76fdf82b81 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:17.853797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ecb142-7e7b-4e6f-9be4-0c76fdf82b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:17.884901 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-80ecb142-7e7b-4e6f-9be4-0c76fdf82b81 None None] MFA Rules not processed for user `d5d2e13a65a04bde9f2e9aa95c421407`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:17.886133 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-80ecb142-7e7b-4e6f-9be4-0c76fdf82b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:17.904497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:17.954425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1336/2694] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:17.965895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:17.967599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:17.990959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:18.027907 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2213e-7f62-451c-b973-b2af4ba69ee6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:18.032620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3551-4722-4814-8834-5bcba13591be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:18.033216 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3551-4722-4814-8834-5bcba13591be service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:18.033494 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3551-4722-4814-8834-5bcba13591be service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:18.033763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd3551-4722-4814-8834-5bcba13591be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:18.042821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd3551-4722-4814-8834-5bcba13591be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:18.044296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd3551-4722-4814-8834-5bcba13591be service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:18.054278 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1359/2695] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:18.157767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ecb142-7e7b-4e6f-9be4-0c76fdf82b81 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:18.159227 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-80ecb142-7e7b-4e6f-9be4-0c76fdf82b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:18.171335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ecb142-7e7b-4e6f-9be4-0c76fdf82b81 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:18.174882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c0fbf-c118-47c1-9b59-c221ba1f5a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:18.175503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c0fbf-c118-47c1-9b59-c221ba1f5a2b service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:18.175780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c0fbf-c118-47c1-9b59-c221ba1f5a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:18.176049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c0fbf-c118-47c1-9b59-c221ba1f5a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:18.231630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c0fbf-c118-47c1-9b59-c221ba1f5a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:18.233000 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c0fbf-c118-47c1-9b59-c221ba1f5a2b service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:18.242010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1337/2696] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:22.975717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3551-4722-4814-8834-5bcba13591be service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:23.038629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd3551-4722-4814-8834-5bcba13591be service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:23.043067 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:23.043684 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:23.043957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:23.044222 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] PATH_INFO: `/v3/users/18ef4c2372c441c8b0c1bebc3cbb0291` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:23.045321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ef4c2372c441c8b0c1bebc3cbb0291)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:23.047271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:23.282066 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ef4c2372c441c8b0c1bebc3cbb0291'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:23.289020 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ef4c2372c441c8b0c1bebc3cbb0291'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:23.289633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ef4c2372c441c8b0c1bebc3cbb0291'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:23.290358 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ef4c2372c441c8b0c1bebc3cbb0291'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:23.306321 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ef4c2372c441c8b0c1bebc3cbb0291'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:23.411164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1360/2697] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:33:22 2022] DELETE /identity/v3/users/18ef4c2372c441c8b0c1bebc3cbb0291 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:23.415774 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27838c7-3cff-49d0-b093-9fa36b7e8ac2 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:23.418726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27838c7-3cff-49d0-b093-9fa36b7e8ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:23.418726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27838c7-3cff-49d0-b093-9fa36b7e8ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:23.435552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-a27838c7-3cff-49d0-b093-9fa36b7e8ac2 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:23.436743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-a27838c7-3cff-49d0-b093-9fa36b7e8ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:23.494660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1338/2698] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:33:23.505104 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:23.506546 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:23.551974 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:23.563284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b60f96-9d4b-4732-ad75-5bddfdd83bf6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:23.567469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:23.568075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:23.568354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:23.568625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:23.577943 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:23.579450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:23.589748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1361/2699] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:23.952031 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27838c7-3cff-49d0-b093-9fa36b7e8ac2 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:23.961554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27838c7-3cff-49d0-b093-9fa36b7e8ac2 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:23.965169 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:23.971334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:23.971334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:23.971334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:23.971334 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:23.977822 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:23.979265 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:23.987710 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1339/2700] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:24.005642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:24.007274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:24.020272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8cce0-c233-42c6-9b89-5f39544be2fc service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:24.025000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:24.025550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:24.025958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:24.026092 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] PATH_INFO: `/v3/projects/c7753568052648008578e422ef01b8a5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:24.027250 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7753568052648008578e422ef01b8a5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:24.029194 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:24.163567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7753568052648008578e422ef01b8a5', 'request_id': 'req-07881ea9-548d-4c3a-999d-1f27cfa86d1e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.251870 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7753568052648008578e422ef01b8a5', 'request_id': 'req-07881ea9-548d-4c3a-999d-1f27cfa86d1e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.258065 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7753568052648008578e422ef01b8a5', 'request_id': 'req-07881ea9-548d-4c3a-999d-1f27cfa86d1e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.270399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] The token cache is being invalidate because project c7753568052648008578e422ef01b8a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:33:24.270689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.271359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.272873 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1362/2701] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:33:24 2022] DELETE /identity/v3/projects/c7753568052648008578e422ef01b8a5 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:24.277100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:24.289723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b19e7-8733-47b8-b77d-9068fa29ae37 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:24.333010 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:24.357918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:24.357918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:24.357918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:24.357918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] PATH_INFO: `/v3/users/d5d2e13a65a04bde9f2e9aa95c421407` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:24.358349 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d2e13a65a04bde9f2e9aa95c421407)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:24.358713 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:24.517506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d2e13a65a04bde9f2e9aa95c421407'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.518372 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d2e13a65a04bde9f2e9aa95c421407'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.525633 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d2e13a65a04bde9f2e9aa95c421407'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.526275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d2e13a65a04bde9f2e9aa95c421407'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.594851 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d77375ff-b9d8-4a02-bfd7-eea05e2d74a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d2e13a65a04bde9f2e9aa95c421407'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:24.614320 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1340/2702] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:33:24 2022] DELETE /identity/v3/users/d5d2e13a65a04bde9f2e9aa95c421407 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:25.008006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:25.009538 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:25.022622 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07881ea9-548d-4c3a-999d-1f27cfa86d1e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:25.080318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:25.080938 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.081213 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.081483 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] PATH_INFO: `/v3/projects/ba69bf6480044db696db7540d51083a4` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.082601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba69bf6480044db696db7540d51083a4)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:25.084558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:25.217554 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba69bf6480044db696db7540d51083a4', 'request_id': 'req-3d943795-dd2f-4329-ba0e-740abf57c11b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:25.302590 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba69bf6480044db696db7540d51083a4', 'request_id': 'req-3d943795-dd2f-4329-ba0e-740abf57c11b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:25.308474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba69bf6480044db696db7540d51083a4', 'request_id': 'req-3d943795-dd2f-4329-ba0e-740abf57c11b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:25.315589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:25.320231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] The token cache is being invalidate because project ba69bf6480044db696db7540d51083a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:33:25.322192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:25.322192 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3d943795-dd2f-4329-ba0e-740abf57c11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:25.322928 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1363/2703] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:33:25 2022] DELETE /identity/v3/projects/ba69bf6480044db696db7540d51083a4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:25.327996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85244d45-07a8-442b-9da4-212daa9151fe None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.328277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85244d45-07a8-442b-9da4-212daa9151fe None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.328548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85244d45-07a8-442b-9da4-212daa9151fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.348391 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-85244d45-07a8-442b-9da4-212daa9151fe None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:25.354995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-85244d45-07a8-442b-9da4-212daa9151fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:25.408375 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1341/2704] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:25.417472 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4126-a45c-4e07-85db-f8ad0a0c983c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.417772 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4126-a45c-4e07-85db-f8ad0a0c983c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.418191 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4126-a45c-4e07-85db-f8ad0a0c983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.438356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c0ae4126-a45c-4e07-85db-f8ad0a0c983c None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:25.439676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c0ae4126-a45c-4e07-85db-f8ad0a0c983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:25.447807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1364/2705] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:25.451911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85244d45-07a8-442b-9da4-212daa9151fe None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:25.461547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85244d45-07a8-442b-9da4-212daa9151fe None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:25.466058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:25.466642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.466917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.467212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.467895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:33:25.468146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:25.469728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:25.481414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1342/2706] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:33:25.486074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae4126-a45c-4e07-85db-f8ad0a0c983c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:25.495293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae4126-a45c-4e07-85db-f8ad0a0c983c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:25.500005 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3120c1-5504-4283-8c9d-91614d531876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:25.500589 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3120c1-5504-4283-8c9d-91614d531876 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.500869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3120c1-5504-4283-8c9d-91614d531876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.501171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3120c1-5504-4283-8c9d-91614d531876 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.501787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3120c1-5504-4283-8c9d-91614d531876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:25.503742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3120c1-5504-4283-8c9d-91614d531876 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:25.566957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1365/2707] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:33:25 2022] POST /identity/v3/projects => generated 387 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:25.571168 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:25.581716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8d0eb-a765-4057-b8eb-d6e6259a6907 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:25.625071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:25.636635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bada644-b02a-4455-8da3-c47a486ad610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:25.637224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bada644-b02a-4455-8da3-c47a486ad610 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.637497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bada644-b02a-4455-8da3-c47a486ad610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.637763 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bada644-b02a-4455-8da3-c47a486ad610 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.638383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bada644-b02a-4455-8da3-c47a486ad610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:25.640311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bada644-b02a-4455-8da3-c47a486ad610 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:25.726467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1343/2708] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:33:25 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:33:25.730656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3120c1-5504-4283-8c9d-91614d531876 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:25.741266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3120c1-5504-4283-8c9d-91614d531876 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:25.746129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13e8c1-3cf5-4f91-8272-2b0f5d06312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:25.746722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13e8c1-3cf5-4f91-8272-2b0f5d06312c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.747041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13e8c1-3cf5-4f91-8272-2b0f5d06312c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.747457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13e8c1-3cf5-4f91-8272-2b0f5d06312c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.748093 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13e8c1-3cf5-4f91-8272-2b0f5d06312c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:25.749737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13e8c1-3cf5-4f91-8272-2b0f5d06312c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:25.759840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1366/2709] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:33:25.763841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bada644-b02a-4455-8da3-c47a486ad610 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:25.774694 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bada644-b02a-4455-8da3-c47a486ad610 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:25.779727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31a851-d645-4624-81be-87498345c387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:25.780333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31a851-d645-4624-81be-87498345c387 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.788797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31a851-d645-4624-81be-87498345c387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.788797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31a851-d645-4624-81be-87498345c387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.788797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31a851-d645-4624-81be-87498345c387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:25.788797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31a851-d645-4624-81be-87498345c387 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:25.794970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1344/2710] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:25.799256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13e8c1-3cf5-4f91-8272-2b0f5d06312c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:25.809347 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13e8c1-3cf5-4f91-8272-2b0f5d06312c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:25.814086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c06e4-4b75-4dc5-b157-b00d3c575022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:25.814726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c06e4-4b75-4dc5-b157-b00d3c575022 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.815039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c06e4-4b75-4dc5-b157-b00d3c575022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.815370 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c06e4-4b75-4dc5-b157-b00d3c575022 admin admin] PATH_INFO: `/v3/projects/b27549ee9d8c4ba0bd5b837a53760287/users/4f6137c06e9b4afa9464abfb9fa43afa/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.830631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c06e4-4b75-4dc5-b157-b00d3c575022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27549ee9d8c4ba0bd5b837a53760287, user_id=4f6137c06e9b4afa9464abfb9fa43afa, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:25.833361 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c06e4-4b75-4dc5-b157-b00d3c575022 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:25.902573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1367/2711] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:25 2022] PUT /identity/v3/projects/b27549ee9d8c4ba0bd5b837a53760287/users/4f6137c06e9b4afa9464abfb9fa43afa/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:25.913536 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d2647-7719-4d9c-8863-886ce9eeff52 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:25.913818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d2647-7719-4d9c-8863-886ce9eeff52 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:25.914084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d2647-7719-4d9c-8863-886ce9eeff52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:25.940856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c08d2647-7719-4d9c-8863-886ce9eeff52 None None] MFA Rules not processed for user `4f6137c06e9b4afa9464abfb9fa43afa`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:25.942115 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c08d2647-7719-4d9c-8863-886ce9eeff52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:26.004812 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1345/2712] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:26.009427 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c06e4-4b75-4dc5-b157-b00d3c575022 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.069850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c06e4-4b75-4dc5-b157-b00d3c575022 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.074751 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65d48-fd1c-48d6-a340-7075188e8020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.075372 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65d48-fd1c-48d6-a340-7075188e8020 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.075651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65d48-fd1c-48d6-a340-7075188e8020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.075921 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65d48-fd1c-48d6-a340-7075188e8020 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.076560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65d48-fd1c-48d6-a340-7075188e8020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.078465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65d48-fd1c-48d6-a340-7075188e8020 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:26.155676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1368/2713] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:33:26 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:26.160007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d2647-7719-4d9c-8863-886ce9eeff52 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.170868 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d2647-7719-4d9c-8863-886ce9eeff52 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.225818 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5713e8-ed31-45e1-9ce1-8e850c853281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.226413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5713e8-ed31-45e1-9ce1-8e850c853281 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.226692 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5713e8-ed31-45e1-9ce1-8e850c853281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.227047 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5713e8-ed31-45e1-9ce1-8e850c853281 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.227673 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5713e8-ed31-45e1-9ce1-8e850c853281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.229583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5713e8-ed31-45e1-9ce1-8e850c853281 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:26.302876 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1346/2714] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:33:26 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:33:26.307086 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65d48-fd1c-48d6-a340-7075188e8020 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.317996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65d48-fd1c-48d6-a340-7075188e8020 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.322973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b65ecb-9cd6-46bf-a0bc-78b4636bcc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.323574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b65ecb-9cd6-46bf-a0bc-78b4636bcc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.323856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b65ecb-9cd6-46bf-a0bc-78b4636bcc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.324128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b65ecb-9cd6-46bf-a0bc-78b4636bcc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.324762 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b65ecb-9cd6-46bf-a0bc-78b4636bcc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.326389 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b65ecb-9cd6-46bf-a0bc-78b4636bcc04 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:26.337463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1369/2715] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:26.341370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5713e8-ed31-45e1-9ce1-8e850c853281 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.351459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5713e8-ed31-45e1-9ce1-8e850c853281 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.356444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543419bd-9a48-4659-96db-c860a194e00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.357121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543419bd-9a48-4659-96db-c860a194e00b admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.357405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543419bd-9a48-4659-96db-c860a194e00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.357681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543419bd-9a48-4659-96db-c860a194e00b admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.358324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543419bd-9a48-4659-96db-c860a194e00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.360036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543419bd-9a48-4659-96db-c860a194e00b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:26.370307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1347/2716] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:26.374329 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b65ecb-9cd6-46bf-a0bc-78b4636bcc04 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.384866 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b65ecb-9cd6-46bf-a0bc-78b4636bcc04 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.389511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbf6e1-8ce8-486b-938d-216713126ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.391033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbf6e1-8ce8-486b-938d-216713126ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.391033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbf6e1-8ce8-486b-938d-216713126ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.391033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbf6e1-8ce8-486b-938d-216713126ad4 admin admin] PATH_INFO: `/v3/projects/04431a0c96874b229b6ff1ed2358feec/users/e48f2c82e1aa4897901336f5ee66ffb4/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.405310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbf6e1-8ce8-486b-938d-216713126ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04431a0c96874b229b6ff1ed2358feec, user_id=e48f2c82e1aa4897901336f5ee66ffb4, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.408044 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbf6e1-8ce8-486b-938d-216713126ad4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:26.473128 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1370/2717] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:26 2022] PUT /identity/v3/projects/04431a0c96874b229b6ff1ed2358feec/users/e48f2c82e1aa4897901336f5ee66ffb4/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:33:26.483593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada37cd-f03e-4710-93b0-7b861ebcaece None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.483877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada37cd-f03e-4710-93b0-7b861ebcaece None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.484147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada37cd-f03e-4710-93b0-7b861ebcaece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.513257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5ada37cd-f03e-4710-93b0-7b861ebcaece None None] MFA Rules not processed for user `e48f2c82e1aa4897901336f5ee66ffb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:26.514385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5ada37cd-f03e-4710-93b0-7b861ebcaece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:26.538599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:26.579884 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1348/2718] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:26.584393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbf6e1-8ce8-486b-938d-216713126ad4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.630911 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:26.647292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbf6e1-8ce8-486b-938d-216713126ad4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.652000 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68b891-df49-41c9-9c88-1ec050c787a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.652599 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68b891-df49-41c9-9c88-1ec050c787a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.652882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68b891-df49-41c9-9c88-1ec050c787a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.653154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68b891-df49-41c9-9c88-1ec050c787a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.653773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68b891-df49-41c9-9c88-1ec050c787a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.655763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68b891-df49-41c9-9c88-1ec050c787a4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:26.743540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1371/2719] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:33:26 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:26.747608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada37cd-f03e-4710-93b0-7b861ebcaece None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.758452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada37cd-f03e-4710-93b0-7b861ebcaece None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.816199 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5bb6f-7675-439b-afc6-a9863d89699b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.816806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5bb6f-7675-439b-afc6-a9863d89699b admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.817087 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5bb6f-7675-439b-afc6-a9863d89699b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.817358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5bb6f-7675-439b-afc6-a9863d89699b admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.817975 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5bb6f-7675-439b-afc6-a9863d89699b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.819946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5bb6f-7675-439b-afc6-a9863d89699b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:26.926644 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1349/2720] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:33:26 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:26.930580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68b891-df49-41c9-9c88-1ec050c787a4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.940843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68b891-df49-41c9-9c88-1ec050c787a4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.945496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8da43-3908-47d2-b9b1-4fcfdaac2bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.946076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8da43-3908-47d2-b9b1-4fcfdaac2bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.946350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8da43-3908-47d2-b9b1-4fcfdaac2bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.946869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8da43-3908-47d2-b9b1-4fcfdaac2bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.947586 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8da43-3908-47d2-b9b1-4fcfdaac2bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.949176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8da43-3908-47d2-b9b1-4fcfdaac2bdb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:26.959559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1372/2721] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:26.963820 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5bb6f-7675-439b-afc6-a9863d89699b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:26.974211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5bb6f-7675-439b-afc6-a9863d89699b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:26.978825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0f139-ad98-4737-b393-466a6eab6096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:26.979497 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f139-ad98-4737-b393-466a6eab6096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:26.979778 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f139-ad98-4737-b393-466a6eab6096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:26.980050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f139-ad98-4737-b393-466a6eab6096 admin admin] PATH_INFO: `/v3/projects/3f9d67b38e3046c8bf916555f5b9c6de/users/4466617aa0bb443aaaca4f21af5f0372/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:26.995187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0f139-ad98-4737-b393-466a6eab6096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9d67b38e3046c8bf916555f5b9c6de, user_id=4466617aa0bb443aaaca4f21af5f0372, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:26.997846 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0f139-ad98-4737-b393-466a6eab6096 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:27.098356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1350/2722] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:26 2022] PUT /identity/v3/projects/3f9d67b38e3046c8bf916555f5b9c6de/users/4466617aa0bb443aaaca4f21af5f0372/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:27.102362 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8da43-3908-47d2-b9b1-4fcfdaac2bdb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:27.112869 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8da43-3908-47d2-b9b1-4fcfdaac2bdb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:27.170153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46849ec6-91ee-4a56-b771-b388b7268a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:27.170915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46849ec6-91ee-4a56-b771-b388b7268a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:27.171223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46849ec6-91ee-4a56-b771-b388b7268a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:27.171492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46849ec6-91ee-4a56-b771-b388b7268a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:27.172126 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46849ec6-91ee-4a56-b771-b388b7268a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:27.173742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46849ec6-91ee-4a56-b771-b388b7268a2a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:27.183366 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1373/2723] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:27.187447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0f139-ad98-4737-b393-466a6eab6096 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:27.197893 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0f139-ad98-4737-b393-466a6eab6096 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:27.202511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:27.203181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:27.203463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:27.203734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] PATH_INFO: `/v3/projects/3f9d67b38e3046c8bf916555f5b9c6de/users/4466617aa0bb443aaaca4f21af5f0372/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:27.206219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9d67b38e3046c8bf916555f5b9c6de, user_id=4466617aa0bb443aaaca4f21af5f0372, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:27.208879 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:27.218804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1351/2724] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:27 2022] PUT /identity/v3/projects/3f9d67b38e3046c8bf916555f5b9c6de/users/4466617aa0bb443aaaca4f21af5f0372/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:27.283270 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c12e-f36b-492d-b876-c0897fe58602 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:27.283579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c12e-f36b-492d-b876-c0897fe58602 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:27.283849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c12e-f36b-492d-b876-c0897fe58602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:27.310720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9731c12e-f36b-492d-b876-c0897fe58602 None None] MFA Rules not processed for user `4466617aa0bb443aaaca4f21af5f0372`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:27.311969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9731c12e-f36b-492d-b876-c0897fe58602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:27.378455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1374/2725] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:27.388946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:27.390627 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:27.448344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12666d7e-6180-4295-8f0e-933e9de0b8af admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:27.451984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c78c1f-2641-4338-812f-e4081d05f7a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:27.452580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c78c1f-2641-4338-812f-e4081d05f7a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:27.452858 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c78c1f-2641-4338-812f-e4081d05f7a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:27.453125 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c78c1f-2641-4338-812f-e4081d05f7a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:27.462330 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c78c1f-2641-4338-812f-e4081d05f7a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:27.463780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c78c1f-2641-4338-812f-e4081d05f7a9 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:27.473647 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1352/2726] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:28.509957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731c12e-f36b-492d-b876-c0897fe58602 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:28.511515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9731c12e-f36b-492d-b876-c0897fe58602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:28.523763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731c12e-f36b-492d-b876-c0897fe58602 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:28.527247 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83332686-6a32-4b5a-b77f-5e4b4d444114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:28.527835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332686-6a32-4b5a-b77f-5e4b4d444114 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:28.528109 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332686-6a32-4b5a-b77f-5e4b4d444114 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:28.528378 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332686-6a32-4b5a-b77f-5e4b4d444114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:28.566648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:28.585386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83332686-6a32-4b5a-b77f-5e4b4d444114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:28.586816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83332686-6a32-4b5a-b77f-5e4b4d444114 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:28.595685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1375/2727] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:33.415358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c78c1f-2641-4338-812f-e4081d05f7a9 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:33.424283 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c78c1f-2641-4338-812f-e4081d05f7a9 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:33.428141 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf954404-9af5-40d2-96c3-4a9e785772b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:33.428729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954404-9af5-40d2-96c3-4a9e785772b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:33.432489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954404-9af5-40d2-96c3-4a9e785772b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:33.432489 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf954404-9af5-40d2-96c3-4a9e785772b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:33.446344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:33.485417 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf954404-9af5-40d2-96c3-4a9e785772b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:33.486841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf954404-9af5-40d2-96c3-4a9e785772b5 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:33.495648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1353/2728] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:37.323729 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83332686-6a32-4b5a-b77f-5e4b4d444114 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:37.384325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83332686-6a32-4b5a-b77f-5e4b4d444114 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:37.388825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:37.389435 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:37.389708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:37.389975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] PATH_INFO: `/v3/users/4f6137c06e9b4afa9464abfb9fa43afa` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:37.391111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f6137c06e9b4afa9464abfb9fa43afa)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:37.393012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:37.573845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f6137c06e9b4afa9464abfb9fa43afa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:37.580664 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f6137c06e9b4afa9464abfb9fa43afa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:37.581354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f6137c06e9b4afa9464abfb9fa43afa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:37.582007 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f6137c06e9b4afa9464abfb9fa43afa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:37.594393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f6137c06e9b4afa9464abfb9fa43afa'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:37.656671 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1376/2729] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:33:37 2022] DELETE /identity/v3/users/4f6137c06e9b4afa9464abfb9fa43afa => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:37.661395 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecf27c-72e3-4d87-b9b6-732e9752ffb5 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:37.662577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecf27c-72e3-4d87-b9b6-732e9752ffb5 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:37.662678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecf27c-72e3-4d87-b9b6-732e9752ffb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:37.690956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-46ecf27c-72e3-4d87-b9b6-732e9752ffb5 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:37.692163 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-46ecf27c-72e3-4d87-b9b6-732e9752ffb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:37.749806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1354/2730] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:37.759593 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:37.760972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:37.819335 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed3283-4985-453a-a25a-2e03e051724a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:37.823503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d66e-4f69-410f-9864-c5776348dbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:37.826097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d66e-4f69-410f-9864-c5776348dbda service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:37.826097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d66e-4f69-410f-9864-c5776348dbda service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:37.826097 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d66e-4f69-410f-9864-c5776348dbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:37.834528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5d66e-4f69-410f-9864-c5776348dbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:37.836012 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5d66e-4f69-410f-9864-c5776348dbda service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:37.841207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:37.846076 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1377/2731] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:33:38.242023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecf27c-72e3-4d87-b9b6-732e9752ffb5 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:38.248150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-46ecf27c-72e3-4d87-b9b6-732e9752ffb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:38.255642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecf27c-72e3-4d87-b9b6-732e9752ffb5 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:38.260915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:38.261550 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:38.261878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:38.262152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] PATH_INFO: `/v3/projects/b27549ee9d8c4ba0bd5b837a53760287` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:38.263397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27549ee9d8c4ba0bd5b837a53760287)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:38.265316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:38.365682 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27549ee9d8c4ba0bd5b837a53760287', 'request_id': 'req-96a751fb-a879-4f01-802c-220606d16dc7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.371039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27549ee9d8c4ba0bd5b837a53760287', 'request_id': 'req-96a751fb-a879-4f01-802c-220606d16dc7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.377208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27549ee9d8c4ba0bd5b837a53760287', 'request_id': 'req-96a751fb-a879-4f01-802c-220606d16dc7'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.507825 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] The token cache is being invalidate because project b27549ee9d8c4ba0bd5b837a53760287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:33:38.508116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.508739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.510232 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1355/2732] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:33:38 2022] DELETE /identity/v3/projects/b27549ee9d8c4ba0bd5b837a53760287 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:38.514452 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d66e-4f69-410f-9864-c5776348dbda service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:38.524283 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d66e-4f69-410f-9864-c5776348dbda service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:38.580792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:38.581433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:38.581728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:38.581991 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] PATH_INFO: `/v3/users/e48f2c82e1aa4897901336f5ee66ffb4` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:38.583149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e48f2c82e1aa4897901336f5ee66ffb4)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:38.585141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:38.756160 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e48f2c82e1aa4897901336f5ee66ffb4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.763368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e48f2c82e1aa4897901336f5ee66ffb4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.764168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e48f2c82e1aa4897901336f5ee66ffb4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.764715 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e48f2c82e1aa4897901336f5ee66ffb4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.777084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e48f2c82e1aa4897901336f5ee66ffb4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:38.885006 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1378/2733] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:33:38 2022] DELETE /identity/v3/users/e48f2c82e1aa4897901336f5ee66ffb4 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:39.253492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:39.255025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:39.268166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a751fb-a879-4f01-802c-220606d16dc7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:39.299894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:39.325286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:39.325903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:39.326179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:39.326448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] PATH_INFO: `/v3/projects/04431a0c96874b229b6ff1ed2358feec` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:39.327564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04431a0c96874b229b6ff1ed2358feec)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:39.329458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:39.416824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04431a0c96874b229b6ff1ed2358feec', 'request_id': 'req-71b69410-98d9-421e-8d31-a7aea22dee4d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.422167 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04431a0c96874b229b6ff1ed2358feec', 'request_id': 'req-71b69410-98d9-421e-8d31-a7aea22dee4d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.428130 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04431a0c96874b229b6ff1ed2358feec', 'request_id': 'req-71b69410-98d9-421e-8d31-a7aea22dee4d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.473302 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] The token cache is being invalidate because project 04431a0c96874b229b6ff1ed2358feec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:33:39.473588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.474208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.475705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1356/2734] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:33:39 2022] DELETE /identity/v3/projects/04431a0c96874b229b6ff1ed2358feec => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:33:39.479835 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:39.481273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:39.494515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58794e24-b1b1-40ac-ae74-e6f462fa314d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:39.552071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:39.552685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:39.552934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:39.553208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] PATH_INFO: `/v3/users/4466617aa0bb443aaaca4f21af5f0372` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:39.554326 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4466617aa0bb443aaaca4f21af5f0372)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:39.556285 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:39.699170 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:39.853611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4466617aa0bb443aaaca4f21af5f0372'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.861054 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4466617aa0bb443aaaca4f21af5f0372'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.861878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4466617aa0bb443aaaca4f21af5f0372'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.862691 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4466617aa0bb443aaaca4f21af5f0372'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.875254 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-50771004-3273-429d-bbe5-8ab35217e619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4466617aa0bb443aaaca4f21af5f0372'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:39.960022 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1379/2735] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:33:39 2022] DELETE /identity/v3/users/4466617aa0bb443aaaca4f21af5f0372 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:40.338766 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:40.340268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:40.362369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69410-98d9-421e-8d31-a7aea22dee4d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:40.418962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:40.419599 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:40.419874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:40.420144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] PATH_INFO: `/v3/projects/3f9d67b38e3046c8bf916555f5b9c6de` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:40.421277 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f9d67b38e3046c8bf916555f5b9c6de)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:40.423202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:40.514249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f9d67b38e3046c8bf916555f5b9c6de', 'request_id': 'req-d8a9a8ab-e156-4838-879e-6d674b070996'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:40.519767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f9d67b38e3046c8bf916555f5b9c6de', 'request_id': 'req-d8a9a8ab-e156-4838-879e-6d674b070996'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:40.526260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f9d67b38e3046c8bf916555f5b9c6de', 'request_id': 'req-d8a9a8ab-e156-4838-879e-6d674b070996'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:40.528786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:40.593267 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] The token cache is being invalidate because project 3f9d67b38e3046c8bf916555f5b9c6de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:33:40.593642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:40.594246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-d8a9a8ab-e156-4838-879e-6d674b070996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:40.595767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1357/2736] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:33:40 2022] DELETE /identity/v3/projects/3f9d67b38e3046c8bf916555f5b9c6de => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:40.602202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c3685-c4b8-4b87-881d-a5e3fb7bccc6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:40.602685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c3685-c4b8-4b87-881d-a5e3fb7bccc6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:40.602800 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c3685-c4b8-4b87-881d-a5e3fb7bccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:40.621371 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8d1c3685-c4b8-4b87-881d-a5e3fb7bccc6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:40.622561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8d1c3685-c4b8-4b87-881d-a5e3fb7bccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:40.679661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1380/2737] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:40.687860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2fcee-d624-4dad-87d4-85193075ca2f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:40.688146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2fcee-d624-4dad-87d4-85193075ca2f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:40.688509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2fcee-d624-4dad-87d4-85193075ca2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:40.709304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-9ea2fcee-d624-4dad-87d4-85193075ca2f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:40.710490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9ea2fcee-d624-4dad-87d4-85193075ca2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:40.718272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1358/2738] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:40.722352 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c3685-c4b8-4b87-881d-a5e3fb7bccc6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:40.732552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c3685-c4b8-4b87-881d-a5e3fb7bccc6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:40.737084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:40.741388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:40.741388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:40.741388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:40.741388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:33:40.741388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:40.741388 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:40.752845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1381/2739] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:33:40.757258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2fcee-d624-4dad-87d4-85193075ca2f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:40.766741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2fcee-d624-4dad-87d4-85193075ca2f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:40.771948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2d822-5353-4a64-adeb-4f7095f1e96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:40.772545 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2d822-5353-4a64-adeb-4f7095f1e96e admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:40.772820 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2d822-5353-4a64-adeb-4f7095f1e96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:40.773086 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2d822-5353-4a64-adeb-4f7095f1e96e admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:40.773718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2d822-5353-4a64-adeb-4f7095f1e96e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:40.775681 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2d822-5353-4a64-adeb-4f7095f1e96e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:40.887784 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1359/2740] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:33:40 2022] POST /identity/v3/projects => generated 391 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:33:40.891957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:40.902405 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e9fb7-78bf-4d3d-ba7b-33645fb421af admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:40.955985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f2ecf-5be2-4a5f-820e-e13ce753a38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:40.956507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f2ecf-5be2-4a5f-820e-e13ce753a38a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:40.956780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f2ecf-5be2-4a5f-820e-e13ce753a38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:40.957048 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f2ecf-5be2-4a5f-820e-e13ce753a38a admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:40.957659 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f2ecf-5be2-4a5f-820e-e13ce753a38a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:40.960230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f2ecf-5be2-4a5f-820e-e13ce753a38a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.062010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1382/2741] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:33:40 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:41.066294 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2d822-5353-4a64-adeb-4f7095f1e96e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.077476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2d822-5353-4a64-adeb-4f7095f1e96e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.082020 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2db5c-56fa-4f12-9d19-59251a0b5d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.083249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2db5c-56fa-4f12-9d19-59251a0b5d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.084249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2db5c-56fa-4f12-9d19-59251a0b5d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.084517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2db5c-56fa-4f12-9d19-59251a0b5d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.085154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2db5c-56fa-4f12-9d19-59251a0b5d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.086741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2db5c-56fa-4f12-9d19-59251a0b5d24 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.097338 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1360/2742] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:41.101471 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f2ecf-5be2-4a5f-820e-e13ce753a38a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.111480 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f2ecf-5be2-4a5f-820e-e13ce753a38a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.116437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62ab8d-ca95-4cea-b513-3b4961ba59f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.117029 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62ab8d-ca95-4cea-b513-3b4961ba59f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.117308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62ab8d-ca95-4cea-b513-3b4961ba59f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.117577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62ab8d-ca95-4cea-b513-3b4961ba59f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.118214 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62ab8d-ca95-4cea-b513-3b4961ba59f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.119840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62ab8d-ca95-4cea-b513-3b4961ba59f1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.129652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1383/2743] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:41.134154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2db5c-56fa-4f12-9d19-59251a0b5d24 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.143755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2db5c-56fa-4f12-9d19-59251a0b5d24 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.148944 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590ae5-bced-4ffd-b848-2b12147bf1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.149565 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590ae5-bced-4ffd-b848-2b12147bf1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.149836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590ae5-bced-4ffd-b848-2b12147bf1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.150101 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590ae5-bced-4ffd-b848-2b12147bf1cf admin admin] PATH_INFO: `/v3/projects/715f92e65a4d4af5a8cc5e26e999c0d3/users/8af37d176b9147a68034cee04a18b57f/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.164064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10590ae5-bced-4ffd-b848-2b12147bf1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=715f92e65a4d4af5a8cc5e26e999c0d3, user_id=8af37d176b9147a68034cee04a18b57f, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.166675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10590ae5-bced-4ffd-b848-2b12147bf1cf admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.233239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1361/2744] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:41 2022] PUT /identity/v3/projects/715f92e65a4d4af5a8cc5e26e999c0d3/users/8af37d176b9147a68034cee04a18b57f/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:41.246081 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b449ec6-d092-4c58-a904-5f8fbfbe55dc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.246374 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b449ec6-d092-4c58-a904-5f8fbfbe55dc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.246651 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b449ec6-d092-4c58-a904-5f8fbfbe55dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.254723 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:41.274443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2b449ec6-d092-4c58-a904-5f8fbfbe55dc None None] MFA Rules not processed for user `8af37d176b9147a68034cee04a18b57f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:41.275685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2b449ec6-d092-4c58-a904-5f8fbfbe55dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:41.338598 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1384/2745] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:41.343238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590ae5-bced-4ffd-b848-2b12147bf1cf admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.404120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590ae5-bced-4ffd-b848-2b12147bf1cf admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.408791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef0a42-ec4b-4d09-9805-2da0599920cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.409383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef0a42-ec4b-4d09-9805-2da0599920cd admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.409663 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef0a42-ec4b-4d09-9805-2da0599920cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.409931 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef0a42-ec4b-4d09-9805-2da0599920cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.410554 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef0a42-ec4b-4d09-9805-2da0599920cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.413113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef0a42-ec4b-4d09-9805-2da0599920cd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.492938 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1362/2746] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:33:41 2022] POST /identity/v3/projects => generated 393 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:41.497071 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b449ec6-d092-4c58-a904-5f8fbfbe55dc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.507887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b449ec6-d092-4c58-a904-5f8fbfbe55dc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.563834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88969331-5ce4-4d6c-a694-d0b1651f10a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.564447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969331-5ce4-4d6c-a694-d0b1651f10a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.564728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969331-5ce4-4d6c-a694-d0b1651f10a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.565002 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969331-5ce4-4d6c-a694-d0b1651f10a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.565618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88969331-5ce4-4d6c-a694-d0b1651f10a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.567564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88969331-5ce4-4d6c-a694-d0b1651f10a1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.651281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1385/2747] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:33:41 2022] POST /identity/v3/users => generated 303 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:41.655290 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef0a42-ec4b-4d09-9805-2da0599920cd admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.665755 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef0a42-ec4b-4d09-9805-2da0599920cd admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.670478 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515f554-0de8-440c-a6f7-48fd5671e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.671100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f554-0de8-440c-a6f7-48fd5671e591 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.671387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f554-0de8-440c-a6f7-48fd5671e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.671750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f554-0de8-440c-a6f7-48fd5671e591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.672483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515f554-0de8-440c-a6f7-48fd5671e591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.674083 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515f554-0de8-440c-a6f7-48fd5671e591 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.685037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1363/2748] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:41.689041 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88969331-5ce4-4d6c-a694-d0b1651f10a1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.699913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88969331-5ce4-4d6c-a694-d0b1651f10a1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.704539 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872441d-be54-4b45-b824-d895e0bff563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.705168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872441d-be54-4b45-b824-d895e0bff563 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.705446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872441d-be54-4b45-b824-d895e0bff563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.705720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872441d-be54-4b45-b824-d895e0bff563 admin admin] PATH_INFO: `/v3/projects/3bf72513bc1a460a9afe7b907014e87e/users/edf8ac2eab92475bb5bc933c9c489cbb/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.720724 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872441d-be54-4b45-b824-d895e0bff563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf72513bc1a460a9afe7b907014e87e, user_id=edf8ac2eab92475bb5bc933c9c489cbb, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.723394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872441d-be54-4b45-b824-d895e0bff563 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.829436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1386/2749] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:41 2022] PUT /identity/v3/projects/3bf72513bc1a460a9afe7b907014e87e/users/edf8ac2eab92475bb5bc933c9c489cbb/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:41.833561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515f554-0de8-440c-a6f7-48fd5671e591 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.844528 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515f554-0de8-440c-a6f7-48fd5671e591 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.900093 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c5f3-bde9-493a-897c-2a80973d6715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.900813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c5f3-bde9-493a-897c-2a80973d6715 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.901103 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c5f3-bde9-493a-897c-2a80973d6715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.901381 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c5f3-bde9-493a-897c-2a80973d6715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.902070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c5f3-bde9-493a-897c-2a80973d6715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.903786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c5f3-bde9-493a-897c-2a80973d6715 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.909197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:41.914348 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1364/2750] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:33:41.918633 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872441d-be54-4b45-b824-d895e0bff563 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:41.929090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872441d-be54-4b45-b824-d895e0bff563 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:41.933769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:41.934404 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.934688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.934988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] PATH_INFO: `/v3/projects/3bf72513bc1a460a9afe7b907014e87e/users/edf8ac2eab92475bb5bc933c9c489cbb/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.937057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf72513bc1a460a9afe7b907014e87e, user_id=edf8ac2eab92475bb5bc933c9c489cbb, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:41.939730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:41.950874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1387/2751] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:41 2022] PUT /identity/v3/projects/3bf72513bc1a460a9afe7b907014e87e/users/edf8ac2eab92475bb5bc933c9c489cbb/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:41.961532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0159-abc6-4855-ae5f-b992ba5a89b8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:41.961843 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0159-abc6-4855-ae5f-b992ba5a89b8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:41.962113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0159-abc6-4855-ae5f-b992ba5a89b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:41.990864 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c15e0159-abc6-4855-ae5f-b992ba5a89b8 None None] MFA Rules not processed for user `edf8ac2eab92475bb5bc933c9c489cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:41.992100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c15e0159-abc6-4855-ae5f-b992ba5a89b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:42.058352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1365/2752] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:33:42.070112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:42.071891 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:42.125722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:42.131385 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7404b72-debe-4b91-8889-adc31b76bf26 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:42.134908 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d475265-f850-4468-9b59-794f518a6454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:42.141836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d475265-f850-4468-9b59-794f518a6454 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:42.141836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d475265-f850-4468-9b59-794f518a6454 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:42.141836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d475265-f850-4468-9b59-794f518a6454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:42.144707 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d475265-f850-4468-9b59-794f518a6454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:42.146147 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d475265-f850-4468-9b59-794f518a6454 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:42.155258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1388/2753] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:33:42.202544 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e0159-abc6-4855-ae5f-b992ba5a89b8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:42.204066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c15e0159-abc6-4855-ae5f-b992ba5a89b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:42.216254 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e0159-abc6-4855-ae5f-b992ba5a89b8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:42.219895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e9753-2dfd-4b05-a0a1-1b4e4f6cf9c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:42.220492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e9753-2dfd-4b05-a0a1-1b4e4f6cf9c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:42.220768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e9753-2dfd-4b05-a0a1-1b4e4f6cf9c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:42.221036 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e9753-2dfd-4b05-a0a1-1b4e4f6cf9c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:42.278735 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e9753-2dfd-4b05-a0a1-1b4e4f6cf9c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:42.280222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e9753-2dfd-4b05-a0a1-1b4e4f6cf9c5 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:42.289547 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1366/2754] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:46.833587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d475265-f850-4468-9b59-794f518a6454 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:46.894246 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d475265-f850-4468-9b59-794f518a6454 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:46.898807 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:46.899463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:46.899748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:46.900025 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] PATH_INFO: `/v3/users/8af37d176b9147a68034cee04a18b57f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:46.901176 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af37d176b9147a68034cee04a18b57f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:46.903151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:47.143802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af37d176b9147a68034cee04a18b57f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.150469 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af37d176b9147a68034cee04a18b57f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.151149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af37d176b9147a68034cee04a18b57f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.151757 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af37d176b9147a68034cee04a18b57f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.163859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af37d176b9147a68034cee04a18b57f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.216702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1389/2755] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:33:46 2022] DELETE /identity/v3/users/8af37d176b9147a68034cee04a18b57f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:47.221216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc923cc-d45f-4872-ba57-80a8397a28fe None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:47.221731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc923cc-d45f-4872-ba57-80a8397a28fe None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:47.221857 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc923cc-d45f-4872-ba57-80a8397a28fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:47.240919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-1cc923cc-d45f-4872-ba57-80a8397a28fe None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:47.242108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1cc923cc-d45f-4872-ba57-80a8397a28fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:47.299219 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1367/2756] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:47.308924 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:47.310315 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:47.367035 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad36f2-4de8-4f6f-baca-e0e83938adb5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:47.370565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0112d4-9e34-44b2-bb02-d94f17517bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:47.371187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0112d4-9e34-44b2-bb02-d94f17517bae service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:47.371467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0112d4-9e34-44b2-bb02-d94f17517bae service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:47.371736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0112d4-9e34-44b2-bb02-d94f17517bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:47.380617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0112d4-9e34-44b2-bb02-d94f17517bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:47.382033 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0112d4-9e34-44b2-bb02-d94f17517bae service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:47.391394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1390/2757] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:47.738802 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc923cc-d45f-4872-ba57-80a8397a28fe None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:47.740292 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1cc923cc-d45f-4872-ba57-80a8397a28fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:47.753147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc923cc-d45f-4872-ba57-80a8397a28fe None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:47.759534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:47.760660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:47.761105 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:47.761605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] PATH_INFO: `/v3/projects/715f92e65a4d4af5a8cc5e26e999c0d3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:47.763243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=715f92e65a4d4af5a8cc5e26e999c0d3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:47.765241 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:47.787706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=100 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:47.856911 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '715f92e65a4d4af5a8cc5e26e999c0d3', 'request_id': 'req-0e489639-9f20-45cb-805f-0310ba6a9fb3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.862768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '715f92e65a4d4af5a8cc5e26e999c0d3', 'request_id': 'req-0e489639-9f20-45cb-805f-0310ba6a9fb3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.869229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '715f92e65a4d4af5a8cc5e26e999c0d3', 'request_id': 'req-0e489639-9f20-45cb-805f-0310ba6a9fb3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.930956 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] The token cache is being invalidate because project 715f92e65a4d4af5a8cc5e26e999c0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:33:47.931286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.931942 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:47.933486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1368/2758] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:33:47 2022] DELETE /identity/v3/projects/715f92e65a4d4af5a8cc5e26e999c0d3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:47.937573 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0112d4-9e34-44b2-bb02-d94f17517bae service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:47.947895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0112d4-9e34-44b2-bb02-d94f17517bae service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:47.955717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:48.003926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:48.004474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:48.004755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:48.005031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] PATH_INFO: `/v3/users/edf8ac2eab92475bb5bc933c9c489cbb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:48.006868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf8ac2eab92475bb5bc933c9c489cbb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:48.008814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:48.185061 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf8ac2eab92475bb5bc933c9c489cbb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.191769 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf8ac2eab92475bb5bc933c9c489cbb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.192428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf8ac2eab92475bb5bc933c9c489cbb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.193604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf8ac2eab92475bb5bc933c9c489cbb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.206166 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b64305c2-4d7f-46a8-a46b-43954b0e7033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf8ac2eab92475bb5bc933c9c489cbb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.268169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1391/2759] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:33:47 2022] DELETE /identity/v3/users/edf8ac2eab92475bb5bc933c9c489cbb => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:48.635753 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:48.638408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:48.650527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e489639-9f20-45cb-805f-0310ba6a9fb3 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:48.709444 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:48.710080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:48.710357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:48.710629 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] PATH_INFO: `/v3/projects/3bf72513bc1a460a9afe7b907014e87e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:48.711781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf72513bc1a460a9afe7b907014e87e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:48.713727 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:48.853378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf72513bc1a460a9afe7b907014e87e', 'request_id': 'req-639dd59a-bd88-4b99-ade5-d13d30c43036'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.858824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf72513bc1a460a9afe7b907014e87e', 'request_id': 'req-639dd59a-bd88-4b99-ade5-d13d30c43036'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.865383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf72513bc1a460a9afe7b907014e87e', 'request_id': 'req-639dd59a-bd88-4b99-ade5-d13d30c43036'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.942566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] The token cache is being invalidate because project 3bf72513bc1a460a9afe7b907014e87e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:33:48.942856 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.943527 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-639dd59a-bd88-4b99-ade5-d13d30c43036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:33:48.945033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1369/2760] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:33:48 2022] DELETE /identity/v3/projects/3bf72513bc1a460a9afe7b907014e87e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:48.950124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d5176-ba7c-4119-9806-1be6e291eee4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:48.950609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d5176-ba7c-4119-9806-1be6e291eee4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:48.950714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d5176-ba7c-4119-9806-1be6e291eee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:48.970185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8a7d5176-ba7c-4119-9806-1be6e291eee4 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:48.971446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8a7d5176-ba7c-4119-9806-1be6e291eee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:49.029676 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1392/2761] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:49.038214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd049fc-c344-4414-9c57-0ce81cb2f901 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.038492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd049fc-c344-4414-9c57-0ce81cb2f901 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.038758 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd049fc-c344-4414-9c57-0ce81cb2f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.058387 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6fd049fc-c344-4414-9c57-0ce81cb2f901 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:49.059630 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6fd049fc-c344-4414-9c57-0ce81cb2f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:49.067541 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1370/2762] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:49.071606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d5176-ba7c-4119-9806-1be6e291eee4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:49.081662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d5176-ba7c-4119-9806-1be6e291eee4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:49.086881 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:49.087510 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.087793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.088069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.088708 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:33:49.089011 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:49.090613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:49.102781 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1393/2763] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:33:49.107108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd049fc-c344-4414-9c57-0ce81cb2f901 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:49.116383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd049fc-c344-4414-9c57-0ce81cb2f901 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:49.121095 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a409e92-0c49-4094-ad4a-f3156f845dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:49.121685 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a409e92-0c49-4094-ad4a-f3156f845dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.121964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a409e92-0c49-4094-ad4a-f3156f845dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.122236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a409e92-0c49-4094-ad4a-f3156f845dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.122863 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a409e92-0c49-4094-ad4a-f3156f845dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:49.124835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a409e92-0c49-4094-ad4a-f3156f845dd7 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:49.235792 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1371/2764] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:33:49 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:49.239903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:49.249532 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7578ed-6904-4996-a0a1-28a4cfcf096b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:49.261954 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:49.304241 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0b4fa-9c3f-439e-b86c-85ee0c806059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:49.304868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0b4fa-9c3f-439e-b86c-85ee0c806059 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.305178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0b4fa-9c3f-439e-b86c-85ee0c806059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.305454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0b4fa-9c3f-439e-b86c-85ee0c806059 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.306100 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0b4fa-9c3f-439e-b86c-85ee0c806059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:49.308103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0b4fa-9c3f-439e-b86c-85ee0c806059 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:49.440207 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1394/2765] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:33:49 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:33:49.445306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a409e92-0c49-4094-ad4a-f3156f845dd7 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:49.455786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a409e92-0c49-4094-ad4a-f3156f845dd7 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:49.460951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82beeb2-a563-44a4-a5b0-dcde05044a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:49.461567 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82beeb2-a563-44a4-a5b0-dcde05044a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.461836 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82beeb2-a563-44a4-a5b0-dcde05044a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.462107 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82beeb2-a563-44a4-a5b0-dcde05044a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.462770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82beeb2-a563-44a4-a5b0-dcde05044a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:49.464388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82beeb2-a563-44a4-a5b0-dcde05044a0d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:49.466352 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:49.474748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1372/2766] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:33:49.478627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0b4fa-9c3f-439e-b86c-85ee0c806059 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:49.488705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0b4fa-9c3f-439e-b86c-85ee0c806059 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:49.493957 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f2509-188f-4997-b14b-e8a58375bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:49.494533 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2509-188f-4997-b14b-e8a58375bb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.494809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2509-188f-4997-b14b-e8a58375bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.495114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2509-188f-4997-b14b-e8a58375bb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.495742 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f2509-188f-4997-b14b-e8a58375bb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:49.497407 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f2509-188f-4997-b14b-e8a58375bb20 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:49.508603 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1395/2767] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 08:33:49.513052 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82beeb2-a563-44a4-a5b0-dcde05044a0d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:49.523295 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82beeb2-a563-44a4-a5b0-dcde05044a0d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:49.528064 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:49.528708 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.528984 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.529284 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] PATH_INFO: `/v3/projects/ce0c15cadcba4b3f8dc657148bc0faf3/users/d84dec7966d14608bfa3aca17feda598/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.543830 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0c15cadcba4b3f8dc657148bc0faf3, user_id=d84dec7966d14608bfa3aca17feda598, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:49.546486 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:49.617272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1373/2768] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:33:49 2022] PUT /identity/v3/projects/ce0c15cadcba4b3f8dc657148bc0faf3/users/d84dec7966d14608bfa3aca17feda598/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:33:49.628785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62782f-53bb-4fa4-9eb0-fc8309974b4e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.629068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62782f-53bb-4fa4-9eb0-fc8309974b4e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.629337 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62782f-53bb-4fa4-9eb0-fc8309974b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.656617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3b62782f-53bb-4fa4-9eb0-fc8309974b4e None None] MFA Rules not processed for user `d84dec7966d14608bfa3aca17feda598`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:33:49.657808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3b62782f-53bb-4fa4-9eb0-fc8309974b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:49.720443 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1396/2769] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:33:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:33:49.732341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:33:49.734146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:33:49.821166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615a6b4-fc53-4b21-851e-cd2f734d74f9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:33:49.825165 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d59601-c6f1-4cf4-b70a-1546b0147566 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e9526306889402c83f2d33b340c2dac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:33:49.825766 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d59601-c6f1-4cf4-b70a-1546b0147566 service glance] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:33:49.826044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d59601-c6f1-4cf4-b70a-1546b0147566 service glance] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:33:49.826405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d59601-c6f1-4cf4-b70a-1546b0147566 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:33:49.834601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d59601-c6f1-4cf4-b70a-1546b0147566 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:33:49.836049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d59601-c6f1-4cf4-b70a-1546b0147566 service glance] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:33:49.844881 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1374/2770] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:33:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:33:59.717224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:33:59.840862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=46,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:11.555344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62782f-53bb-4fa4-9eb0-fc8309974b4e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:11.557590 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3b62782f-53bb-4fa4-9eb0-fc8309974b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:11.636809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62782f-53bb-4fa4-9eb0-fc8309974b4e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:11.642220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:11.642822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:11.643131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:11.643403 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:11.645105 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:11.703461 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:11.704914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:11.713653 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1397/2771] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:34:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:34:14.955186 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd84b-e8b9-419b-903d-a097ec8ec4a8 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:14.955642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd84b-e8b9-419b-903d-a097ec8ec4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:14.958374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd84b-e8b9-419b-903d-a097ec8ec4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:14.975576 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f9cbd84b-e8b9-419b-903d-a097ec8ec4a8 None None] MFA Rules not processed for user `db8b5a8edcc34d32a3e9fad54d33cbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:34:14.976698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f9cbd84b-e8b9-419b-903d-a097ec8ec4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:15.030654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1375/2772] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:34:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:34:15.356831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:15.358310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:15.415266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbd84b-e8b9-419b-903d-a097ec8ec4a8 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:15.417541 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf449efc-a4f6-4066-a13e-e8b6d310a7bd service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:15.421846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7918a3-8df2-4815-9648-81bdc28e539e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:15.422455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7918a3-8df2-4815-9648-81bdc28e539e service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:15.422736 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7918a3-8df2-4815-9648-81bdc28e539e service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:15.423046 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7918a3-8df2-4815-9648-81bdc28e539e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:15.424750 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4e7918a3-8df2-4815-9648-81bdc28e539e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:15.435052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7918a3-8df2-4815-9648-81bdc28e539e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:15.436722 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7918a3-8df2-4815-9648-81bdc28e539e service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:15.451264 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1398/2773] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:34:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:34:15.480593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbd84b-e8b9-419b-903d-a097ec8ec4a8 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:15.485676 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:15.486304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:15.486586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:15.486861 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] PATH_INFO: `/v3/users/d84dec7966d14608bfa3aca17feda598` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:15.488042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84dec7966d14608bfa3aca17feda598)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:15.490030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:15.683399 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84dec7966d14608bfa3aca17feda598'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:15.684056 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84dec7966d14608bfa3aca17feda598'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:15.691634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84dec7966d14608bfa3aca17feda598'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:15.692278 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84dec7966d14608bfa3aca17feda598'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:15.783228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84dec7966d14608bfa3aca17feda598'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:15.801353 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1376/2774] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:34:15 2022] DELETE /identity/v3/users/d84dec7966d14608bfa3aca17feda598 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:34:15.805865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2de6a-ff7f-4ecb-9745-a03e28fed1e6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:15.806141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2de6a-ff7f-4ecb-9745-a03e28fed1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:15.806447 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2de6a-ff7f-4ecb-9745-a03e28fed1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:15.825827 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b1a2de6a-ff7f-4ecb-9745-a03e28fed1e6 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:34:15.827032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b1a2de6a-ff7f-4ecb-9745-a03e28fed1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:15.862993 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:15.888540 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1399/2775] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:34:15.898562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:15.900001 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:15.928332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:15.958184 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae285481-eb7a-4ef1-9c78-b077cfa87686 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:15.961997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:15.962601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:15.962883 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:15.963183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:15.972296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:15.973726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:15.983728 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1377/2776] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:34:16.144608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2de6a-ff7f-4ecb-9745-a03e28fed1e6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:16.146823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b1a2de6a-ff7f-4ecb-9745-a03e28fed1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:16.268172 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2de6a-ff7f-4ecb-9745-a03e28fed1e6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:16.273621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9edc5-9f01-402a-aca5-bf48ab5d249d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:16.274224 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9edc5-9f01-402a-aca5-bf48ab5d249d service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:16.274501 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9edc5-9f01-402a-aca5-bf48ab5d249d service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:16.274767 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9edc5-9f01-402a-aca5-bf48ab5d249d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:16.276520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f2a9edc5-9f01-402a-aca5-bf48ab5d249d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:16.289124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9edc5-9f01-402a-aca5-bf48ab5d249d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:16.290843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9edc5-9f01-402a-aca5-bf48ab5d249d service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:16.303425 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1400/2777] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:34:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:34:16.493458 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:16.495002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:16.511562 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802edcf-97d4-4cb2-925c-1d565458cddf service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:16.513257 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:16.513974 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:16.514335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:16.514684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] PATH_INFO: `/v3/projects/ce0c15cadcba4b3f8dc657148bc0faf3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:16.515996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce0c15cadcba4b3f8dc657148bc0faf3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:16.518537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:16.657570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce0c15cadcba4b3f8dc657148bc0faf3', 'request_id': 'req-e343fb89-da2d-4491-b664-42ebc24f9e63'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:16.663048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce0c15cadcba4b3f8dc657148bc0faf3', 'request_id': 'req-e343fb89-da2d-4491-b664-42ebc24f9e63'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:16.675340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce0c15cadcba4b3f8dc657148bc0faf3', 'request_id': 'req-e343fb89-da2d-4491-b664-42ebc24f9e63'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:16.743258 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] The token cache is being invalidate because project ce0c15cadcba4b3f8dc657148bc0faf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:34:16.743553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:16.744207 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-e343fb89-da2d-4491-b664-42ebc24f9e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:16.745739 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1378/2778] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:34:16 2022] DELETE /identity/v3/projects/ce0c15cadcba4b3f8dc657148bc0faf3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:34:16.751654 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b190b9-5778-48e3-ade1-c345dab72adc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:16.752119 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b190b9-5778-48e3-ade1-c345dab72adc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:16.752249 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b190b9-5778-48e3-ade1-c345dab72adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:16.772877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f2b190b9-5778-48e3-ade1-c345dab72adc None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:34:16.773998 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f2b190b9-5778-48e3-ade1-c345dab72adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:16.837842 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1401/2779] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:34:16.846415 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193b2e8-bf7c-40c8-946b-d05bfd6828bf None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:16.846797 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193b2e8-bf7c-40c8-946b-d05bfd6828bf None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:16.847146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193b2e8-bf7c-40c8-946b-d05bfd6828bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:16.868376 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-5193b2e8-bf7c-40c8-946b-d05bfd6828bf None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:34:16.869595 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-5193b2e8-bf7c-40c8-946b-d05bfd6828bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:16.877776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1379/2780] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:34:16.882162 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b190b9-5778-48e3-ade1-c345dab72adc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:16.892681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b190b9-5778-48e3-ade1-c345dab72adc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:16.897467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:16.905799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:16.905799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:16.905799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:16.905799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:34:16.905799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:16.905799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:16.913446 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1402/2781] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:34:16.917860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193b2e8-bf7c-40c8-946b-d05bfd6828bf None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:16.927432 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193b2e8-bf7c-40c8-946b-d05bfd6828bf None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:16.932563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e68d6-b192-426e-968f-e40dfa57a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:16.934356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e68d6-b192-426e-968f-e40dfa57a8bc admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:16.934356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e68d6-b192-426e-968f-e40dfa57a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:16.934356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e68d6-b192-426e-968f-e40dfa57a8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:16.934356 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e68d6-b192-426e-968f-e40dfa57a8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:16.936272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e68d6-b192-426e-968f-e40dfa57a8bc admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:17.024225 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1380/2782] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:34:16 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:34:17.029360 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:17.039308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6884c-ba21-4dc7-9ae8-855f5e560e04 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:17.090621 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:17.096774 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569ab5c-3d0b-4aa3-9177-78128b9e9264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:17.097376 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569ab5c-3d0b-4aa3-9177-78128b9e9264 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:17.097650 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569ab5c-3d0b-4aa3-9177-78128b9e9264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:17.097917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569ab5c-3d0b-4aa3-9177-78128b9e9264 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:17.098536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569ab5c-3d0b-4aa3-9177-78128b9e9264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:17.100475 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569ab5c-3d0b-4aa3-9177-78128b9e9264 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:17.218382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1403/2783] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:34:17 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:34:17.222587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e68d6-b192-426e-968f-e40dfa57a8bc admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:17.233059 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e68d6-b192-426e-968f-e40dfa57a8bc admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:17.237736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7d481-f41e-4204-bb3a-a4ccb692291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:17.238326 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d481-f41e-4204-bb3a-a4ccb692291f admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:17.238646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d481-f41e-4204-bb3a-a4ccb692291f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:17.238918 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d481-f41e-4204-bb3a-a4ccb692291f admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:17.239620 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7d481-f41e-4204-bb3a-a4ccb692291f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:17.241249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7d481-f41e-4204-bb3a-a4ccb692291f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:17.252842 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1381/2784] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:34:17.257106 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569ab5c-3d0b-4aa3-9177-78128b9e9264 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:17.267604 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569ab5c-3d0b-4aa3-9177-78128b9e9264 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:17.272208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88a637-dc84-4a45-bbbd-3ae3f01cc9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:17.272797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a637-dc84-4a45-bbbd-3ae3f01cc9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:17.273083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a637-dc84-4a45-bbbd-3ae3f01cc9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:17.273386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a637-dc84-4a45-bbbd-3ae3f01cc9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:17.274024 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88a637-dc84-4a45-bbbd-3ae3f01cc9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:17.275756 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88a637-dc84-4a45-bbbd-3ae3f01cc9dc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:17.286824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1404/2785] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:34:17.291726 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7d481-f41e-4204-bb3a-a4ccb692291f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:17.301939 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7d481-f41e-4204-bb3a-a4ccb692291f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:17.308397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:17.309053 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:17.309335 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:17.309653 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] PATH_INFO: `/v3/projects/cd50fa884143493d94ee526dca007ccc/users/6b63e5e0726a4ab1a2a791262a0a7218/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:17.330456 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd50fa884143493d94ee526dca007ccc, user_id=6b63e5e0726a4ab1a2a791262a0a7218, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:17.333208 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:17.428244 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1382/2786] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:34:17 2022] PUT /identity/v3/projects/cd50fa884143493d94ee526dca007ccc/users/6b63e5e0726a4ab1a2a791262a0a7218/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:34:17.442215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5bbbe-0cf9-4607-88ea-8d8b65b3e74a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:17.442496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5bbbe-0cf9-4607-88ea-8d8b65b3e74a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:17.442766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5bbbe-0cf9-4607-88ea-8d8b65b3e74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:17.479643 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-59a5bbbe-0cf9-4607-88ea-8d8b65b3e74a None None] MFA Rules not processed for user `6b63e5e0726a4ab1a2a791262a0a7218`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:34:17.480874 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-59a5bbbe-0cf9-4607-88ea-8d8b65b3e74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:17.551990 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1405/2787] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:34:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:34:17.563917 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:17.565770 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:17.621502 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:17.626634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2d11a-6660-457d-b40c-7839bdcf0095 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:17.630433 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429eabf-2975-4f02-9794-ede9793a6e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:17.631041 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429eabf-2975-4f02-9794-ede9793a6e03 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:17.631319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429eabf-2975-4f02-9794-ede9793a6e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:17.631612 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429eabf-2975-4f02-9794-ede9793a6e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:17.641023 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429eabf-2975-4f02-9794-ede9793a6e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:17.642448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429eabf-2975-4f02-9794-ede9793a6e03 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:17.662085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1383/2788] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:34:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:34:17.678318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5bbbe-0cf9-4607-88ea-8d8b65b3e74a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:17.743949 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5bbbe-0cf9-4607-88ea-8d8b65b3e74a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:17.750043 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:17.750673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:17.751168 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:17.751286 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] PATH_INFO: `/v3/users/6b63e5e0726a4ab1a2a791262a0a7218` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:17.752408 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b63e5e0726a4ab1a2a791262a0a7218)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:17.754343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:17.956837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b63e5e0726a4ab1a2a791262a0a7218'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:17.964303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b63e5e0726a4ab1a2a791262a0a7218'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:17.964967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b63e5e0726a4ab1a2a791262a0a7218'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:17.965580 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b63e5e0726a4ab1a2a791262a0a7218'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:17.973549 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:17.978206 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b63e5e0726a4ab1a2a791262a0a7218'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:18.097287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1406/2789] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:34:17 2022] DELETE /identity/v3/users/6b63e5e0726a4ab1a2a791262a0a7218 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:34:18.101814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc6ad-3a69-47c7-b20c-5faf619a7a26 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:18.102316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc6ad-3a69-47c7-b20c-5faf619a7a26 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:18.102423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc6ad-3a69-47c7-b20c-5faf619a7a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:18.121564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-88afc6ad-3a69-47c7-b20c-5faf619a7a26 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:34:18.122841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-88afc6ad-3a69-47c7-b20c-5faf619a7a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:18.181734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1384/2790] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:34:18.191489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:18.192895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:18.250766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64f9bb-4ddc-4089-bdde-46f7b543a438 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:18.254266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:18.254855 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:18.255158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:18.255429 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:18.264865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:18.266281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:18.275419 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1407/2791] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:34:19.000743 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afc6ad-3a69-47c7-b20c-5faf619a7a26 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:19.002248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-88afc6ad-3a69-47c7-b20c-5faf619a7a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:19.014470 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afc6ad-3a69-47c7-b20c-5faf619a7a26 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:19.019621 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:19.020268 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:19.020531 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:19.020806 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] PATH_INFO: `/v3/projects/cd50fa884143493d94ee526dca007ccc` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:19.022028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd50fa884143493d94ee526dca007ccc)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:19.024763 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:19.124303 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd50fa884143493d94ee526dca007ccc', 'request_id': 'req-2cfa5bd6-cc31-414c-b684-c735f6948060'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:19.129914 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd50fa884143493d94ee526dca007ccc', 'request_id': 'req-2cfa5bd6-cc31-414c-b684-c735f6948060'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:19.136212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd50fa884143493d94ee526dca007ccc', 'request_id': 'req-2cfa5bd6-cc31-414c-b684-c735f6948060'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:19.183048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] The token cache is being invalidate because project cd50fa884143493d94ee526dca007ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:34:19.183342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:19.183970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-2cfa5bd6-cc31-414c-b684-c735f6948060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:34:19.185535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1385/2792] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:34:18 2022] DELETE /identity/v3/projects/cd50fa884143493d94ee526dca007ccc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:34:28.271324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=30,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:29.179189 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:35.057212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:35.058840 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:35.181437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760c0a7-f006-42c2-adc0-f0cf91371886 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:35.248259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:35.248876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:35.249151 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:35.249419 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:35.251252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:35.264463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:35.266050 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:35.277018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1408/2793] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:34:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:34:35.704729 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113e7f1-bc08-440c-8163-ae7ac37b605f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:35.705238 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113e7f1-bc08-440c-8163-ae7ac37b605f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:35.705346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113e7f1-bc08-440c-8163-ae7ac37b605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:35.724002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-8113e7f1-bc08-440c-8163-ae7ac37b605f None None] MFA Rules not processed for user `d780debf950447b983dc94edfc681a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:34:35.725187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8113e7f1-bc08-440c-8163-ae7ac37b605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:35.781611 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1386/2794] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:34:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:34:40.881220 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:34:40.883877 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:40.899124 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3310df5-0892-4ef6-a92e-5a31d9790fe8 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:34:40.905034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:34:40.905760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:34:40.906146 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:34:40.906535 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:34:40.908656 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:34:40.924849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:34:40.924849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:34:40.935395 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1409/2795] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:34:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:34:45.773988 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=25 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:34:50.930692 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:20.571228 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113e7f1-bc08-440c-8163-ae7ac37b605f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:20.572675 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-8113e7f1-bc08-440c-8163-ae7ac37b605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:20.662061 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113e7f1-bc08-440c-8163-ae7ac37b605f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:20.667287 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:20.667896 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:20.668182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:20.668455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:20.670259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:20.681990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:20.683606 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:20.694048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1387/2796] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:28.399131 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:28.400584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:28.477103 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a78cb1-0dda-4aa2-a72a-55932fc6cfa6 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:28.480696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:28.481287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:28.481563 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:28.481830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:28.483521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:28.564587 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:28.565933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:28.584863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1410/2797] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:30.689793 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=41 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:31.789781 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:31.802131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd670d4-0ced-4886-ba7a-df766bf21a54 service placement] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:31.807380 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:31.808034 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:31.808337 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:31.808609 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:31.810331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:31.822455 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:31.824222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:31.834583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1388/2798] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:31.924327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:31.925752 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:31.939178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da7952-9cfc-4e20-8acf-85a0c0b0c558 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:31.943926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:31.944584 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:31.944865 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:31.945136 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] PATH_INFO: `/v3/users/d780debf950447b983dc94edfc681a68` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:31.946252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d780debf950447b983dc94edfc681a68)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:31.948497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:32.115909 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd780debf950447b983dc94edfc681a68'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.122704 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd780debf950447b983dc94edfc681a68'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.123397 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd780debf950447b983dc94edfc681a68'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.124018 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd780debf950447b983dc94edfc681a68'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.125134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:32.136258 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d10efd0a-2be3-43c1-9082-d04c010cb849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd780debf950447b983dc94edfc681a68'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.233933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1411/2799] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:36:31 2022] DELETE /identity/v3/users/d780debf950447b983dc94edfc681a68 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:32.522191 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:32.534240 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7fc3-fab9-4fa5-bc13-642b833bc80e service placement] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:32.601496 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:32.602150 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:32.602427 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:32.602701 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] PATH_INFO: `/v3/projects/1367659f7d6a404393adca1c5d713c99` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:32.604015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1367659f7d6a404393adca1c5d713c99)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:32.605965 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:32.718091 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1367659f7d6a404393adca1c5d713c99', 'request_id': 'req-f510e85e-5f15-4986-a2b9-893aa3b5c06e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.723601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1367659f7d6a404393adca1c5d713c99', 'request_id': 'req-f510e85e-5f15-4986-a2b9-893aa3b5c06e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.731465 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1367659f7d6a404393adca1c5d713c99', 'request_id': 'req-f510e85e-5f15-4986-a2b9-893aa3b5c06e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.830674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] The token cache is being invalidate because project 1367659f7d6a404393adca1c5d713c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:36:32.831374 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.831679 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f510e85e-5f15-4986-a2b9-893aa3b5c06e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:32.833245 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1389/2800] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:36:32 2022] DELETE /identity/v3/projects/1367659f7d6a404393adca1c5d713c99 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:32.838271 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88e5b-7942-4e04-b457-b1840b89cd8f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:32.841134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88e5b-7942-4e04-b457-b1840b89cd8f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:32.841134 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88e5b-7942-4e04-b457-b1840b89cd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:32.896420 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e4e88e5b-7942-4e04-b457-b1840b89cd8f None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:32.897624 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e4e88e5b-7942-4e04-b457-b1840b89cd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:32.959212 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1412/2801] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:32.967071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2867-e545-4763-93cb-6d7f13a39bf0 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:32.967360 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2867-e545-4763-93cb-6d7f13a39bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:32.967659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2867-e545-4763-93cb-6d7f13a39bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:32.987377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b05e2867-e545-4763-93cb-6d7f13a39bf0 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:32.988646 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b05e2867-e545-4763-93cb-6d7f13a39bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:32.996537 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1390/2802] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:33.001340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e88e5b-7942-4e04-b457-b1840b89cd8f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.010806 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e88e5b-7942-4e04-b457-b1840b89cd8f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.015377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.015959 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.016230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.016492 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.017117 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:36:33.017418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.019010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.030083 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1413/2803] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:36:33.034138 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e2867-e545-4763-93cb-6d7f13a39bf0 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.043341 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e2867-e545-4763-93cb-6d7f13a39bf0 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.048507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06bd6d-4cfe-479b-8540-23d53db12b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.049092 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06bd6d-4cfe-479b-8540-23d53db12b7b admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.049369 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06bd6d-4cfe-479b-8540-23d53db12b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.049642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06bd6d-4cfe-479b-8540-23d53db12b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.050259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06bd6d-4cfe-479b-8540-23d53db12b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.052181 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06bd6d-4cfe-479b-8540-23d53db12b7b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.163085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1391/2804] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:36:33 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:33.167148 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.177577 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529bb4-7e39-41dc-8f9b-2db4472c636f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.233149 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30947da-1f37-4bca-a65a-f1b581ec62f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.233667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30947da-1f37-4bca-a65a-f1b581ec62f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.233939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30947da-1f37-4bca-a65a-f1b581ec62f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.234234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30947da-1f37-4bca-a65a-f1b581ec62f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.234846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30947da-1f37-4bca-a65a-f1b581ec62f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.236773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30947da-1f37-4bca-a65a-f1b581ec62f8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.299741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1414/2805] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:36:33 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:33.304044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06bd6d-4cfe-479b-8540-23d53db12b7b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.314480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06bd6d-4cfe-479b-8540-23d53db12b7b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.319214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737e5a9-be96-4ef6-abbb-6fb681ea2e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.319799 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e5a9-be96-4ef6-abbb-6fb681ea2e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.320085 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e5a9-be96-4ef6-abbb-6fb681ea2e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.320368 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737e5a9-be96-4ef6-abbb-6fb681ea2e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.321032 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737e5a9-be96-4ef6-abbb-6fb681ea2e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.322608 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737e5a9-be96-4ef6-abbb-6fb681ea2e7b admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.332698 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1392/2806] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:33.336509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30947da-1f37-4bca-a65a-f1b581ec62f8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.346825 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30947da-1f37-4bca-a65a-f1b581ec62f8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.351688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f60ea8-2ec6-4870-ac92-dfbf8589b764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.352273 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60ea8-2ec6-4870-ac92-dfbf8589b764 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.352553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60ea8-2ec6-4870-ac92-dfbf8589b764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.352823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f60ea8-2ec6-4870-ac92-dfbf8589b764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.353454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f60ea8-2ec6-4870-ac92-dfbf8589b764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.355142 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f60ea8-2ec6-4870-ac92-dfbf8589b764 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.366287 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1415/2807] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:33.370323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737e5a9-be96-4ef6-abbb-6fb681ea2e7b admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.380308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737e5a9-be96-4ef6-abbb-6fb681ea2e7b admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.384966 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021ab51-cc33-474d-89b6-649882da6701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.385605 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021ab51-cc33-474d-89b6-649882da6701 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.385882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021ab51-cc33-474d-89b6-649882da6701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.386157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021ab51-cc33-474d-89b6-649882da6701 admin admin] PATH_INFO: `/v3/projects/8c6edf6ccbec416bafead82c1aced733/users/be417fbed3354516934e91d6171777fb/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.400747 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021ab51-cc33-474d-89b6-649882da6701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6edf6ccbec416bafead82c1aced733, user_id=be417fbed3354516934e91d6171777fb, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.403961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021ab51-cc33-474d-89b6-649882da6701 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.487089 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1393/2808] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:33 2022] PUT /identity/v3/projects/8c6edf6ccbec416bafead82c1aced733/users/be417fbed3354516934e91d6171777fb/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:33.498588 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b48aef-be42-428e-889a-9b6c34100f48 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.498879 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b48aef-be42-428e-889a-9b6c34100f48 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.499209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b48aef-be42-428e-889a-9b6c34100f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.519057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:33.527536 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-22b48aef-be42-428e-889a-9b6c34100f48 None None] MFA Rules not processed for user `be417fbed3354516934e91d6171777fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:33.528749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-22b48aef-be42-428e-889a-9b6c34100f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:33.591863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1416/2809] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:33 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:33.596504 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021ab51-cc33-474d-89b6-649882da6701 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.602180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:33.657481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021ab51-cc33-474d-89b6-649882da6701 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.662239 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41107dd1-4c03-4546-a584-b97c800022dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.662841 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41107dd1-4c03-4546-a584-b97c800022dd admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.663149 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41107dd1-4c03-4546-a584-b97c800022dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.663423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41107dd1-4c03-4546-a584-b97c800022dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.664040 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41107dd1-4c03-4546-a584-b97c800022dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.665940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41107dd1-4c03-4546-a584-b97c800022dd admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.763408 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1394/2810] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:36:33 2022] POST /identity/v3/projects => generated 401 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:33.767569 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b48aef-be42-428e-889a-9b6c34100f48 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.778244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b48aef-be42-428e-889a-9b6c34100f48 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.834795 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4849f-e970-49ae-a41f-766f05e7c98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.835428 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4849f-e970-49ae-a41f-766f05e7c98c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.835705 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4849f-e970-49ae-a41f-766f05e7c98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.835972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4849f-e970-49ae-a41f-766f05e7c98c admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.836601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4849f-e970-49ae-a41f-766f05e7c98c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.838499 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4849f-e970-49ae-a41f-766f05e7c98c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.932642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1417/2811] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:36:33 2022] POST /identity/v3/users => generated 307 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:36:33.936791 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41107dd1-4c03-4546-a584-b97c800022dd admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.947462 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41107dd1-4c03-4546-a584-b97c800022dd admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.952179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba4ab9-252d-4d4b-afba-6af2d0ab3c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.953026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba4ab9-252d-4d4b-afba-6af2d0ab3c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.953306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba4ab9-252d-4d4b-afba-6af2d0ab3c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.953577 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba4ab9-252d-4d4b-afba-6af2d0ab3c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:33.954217 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba4ab9-252d-4d4b-afba-6af2d0ab3c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:33.955874 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba4ab9-252d-4d4b-afba-6af2d0ab3c81 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:33.967285 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1395/2812] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:33.971852 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4849f-e970-49ae-a41f-766f05e7c98c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:33.982712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4849f-e970-49ae-a41f-766f05e7c98c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:33.987436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01ff3d-fff2-4581-a1d0-770fcd16ad05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:33.988073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ff3d-fff2-4581-a1d0-770fcd16ad05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:33.988350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ff3d-fff2-4581-a1d0-770fcd16ad05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:33.988625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ff3d-fff2-4581-a1d0-770fcd16ad05 admin admin] PATH_INFO: `/v3/projects/821b99792adc41e889e74feec32ecdbe/users/de1457f45c10415eb6d93949e40a9d9e/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:34.003496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01ff3d-fff2-4581-a1d0-770fcd16ad05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=821b99792adc41e889e74feec32ecdbe, user_id=de1457f45c10415eb6d93949e40a9d9e, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:34.006140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01ff3d-fff2-4581-a1d0-770fcd16ad05 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:34.068562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1418/2813] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:33 2022] PUT /identity/v3/projects/821b99792adc41e889e74feec32ecdbe/users/de1457f45c10415eb6d93949e40a9d9e/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:34.072423 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba4ab9-252d-4d4b-afba-6af2d0ab3c81 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:34.082768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba4ab9-252d-4d4b-afba-6af2d0ab3c81 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:34.139336 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158c7f8-7d40-46d8-8033-be2cc05b83af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:34.139937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158c7f8-7d40-46d8-8033-be2cc05b83af admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:34.140223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158c7f8-7d40-46d8-8033-be2cc05b83af admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:34.140495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158c7f8-7d40-46d8-8033-be2cc05b83af admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:34.141129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158c7f8-7d40-46d8-8033-be2cc05b83af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:34.142716 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158c7f8-7d40-46d8-8033-be2cc05b83af admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:34.152563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1396/2814] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:34.162484 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01ff3d-fff2-4581-a1d0-770fcd16ad05 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:34.173882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01ff3d-fff2-4581-a1d0-770fcd16ad05 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:34.178555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:34.179217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:34.179498 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:34.179771 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] PATH_INFO: `/v3/projects/821b99792adc41e889e74feec32ecdbe/users/de1457f45c10415eb6d93949e40a9d9e/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:34.181698 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=821b99792adc41e889e74feec32ecdbe, user_id=de1457f45c10415eb6d93949e40a9d9e, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:34.184450 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:34.194410 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1419/2815] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:34 2022] PUT /identity/v3/projects/821b99792adc41e889e74feec32ecdbe/users/de1457f45c10415eb6d93949e40a9d9e/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:34.204525 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1f882-2beb-4d91-872f-0acc667ae964 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:34.204805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1f882-2beb-4d91-872f-0acc667ae964 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:34.205072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1f882-2beb-4d91-872f-0acc667ae964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:34.232665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b2d1f882-2beb-4d91-872f-0acc667ae964 None None] MFA Rules not processed for user `de1457f45c10415eb6d93949e40a9d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:34.233849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b2d1f882-2beb-4d91-872f-0acc667ae964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:34.297894 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:34.301538 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1397/2816] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:34 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:34.312728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:34.314448 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:34.372208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f7408-3dbb-48f9-bcab-90f63c097e6f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:34.375823 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a183377-3694-4053-b2d6-098f467d3f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:34.376409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183377-3694-4053-b2d6-098f467d3f68 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:34.376681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183377-3694-4053-b2d6-098f467d3f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:34.376944 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a183377-3694-4053-b2d6-098f467d3f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:34.382712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:34.386256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a183377-3694-4053-b2d6-098f467d3f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:34.388235 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a183377-3694-4053-b2d6-098f467d3f68 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:34.397412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1420/2817] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:34 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:34.587592 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1f882-2beb-4d91-872f-0acc667ae964 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:34.597445 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1f882-2beb-4d91-872f-0acc667ae964 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:34.600947 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cce40a-d8b2-4636-a1e3-9e367fcef691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:34.601534 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce40a-d8b2-4636-a1e3-9e367fcef691 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:34.601807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce40a-d8b2-4636-a1e3-9e367fcef691 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:34.602074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce40a-d8b2-4636-a1e3-9e367fcef691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:34.659203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cce40a-d8b2-4636-a1e3-9e367fcef691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:34.660888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cce40a-d8b2-4636-a1e3-9e367fcef691 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:34.670388 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1398/2818] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:39.557183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a183377-3694-4053-b2d6-098f467d3f68 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:39.566463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a183377-3694-4053-b2d6-098f467d3f68 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:39.570095 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7551e3e2-c2ec-4eb1-9eb8-b321eab1adfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:39.570694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551e3e2-c2ec-4eb1-9eb8-b321eab1adfe service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:39.571032 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551e3e2-c2ec-4eb1-9eb8-b321eab1adfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:39.571269 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551e3e2-c2ec-4eb1-9eb8-b321eab1adfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:39.573057 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7551e3e2-c2ec-4eb1-9eb8-b321eab1adfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:39.634430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551e3e2-c2ec-4eb1-9eb8-b321eab1adfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:39.635859 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7551e3e2-c2ec-4eb1-9eb8-b321eab1adfe service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:39.645319 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1421/2819] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:39.719575 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cce40a-d8b2-4636-a1e3-9e367fcef691 service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:39.783829 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cce40a-d8b2-4636-a1e3-9e367fcef691 service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:39.788229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:39.788978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:39.789229 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:39.789503 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] PATH_INFO: `/v3/users/be417fbed3354516934e91d6171777fb` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:39.790671 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be417fbed3354516934e91d6171777fb)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:39.792684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:40.009013 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be417fbed3354516934e91d6171777fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.009920 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be417fbed3354516934e91d6171777fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.018322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be417fbed3354516934e91d6171777fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.019026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be417fbed3354516934e91d6171777fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.086849 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be417fbed3354516934e91d6171777fb'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.106033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1399/2820] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:36:39 2022] DELETE /identity/v3/users/be417fbed3354516934e91d6171777fb => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:40.110609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b274df5-eeba-4c63-b4cd-9aa489cb9ee7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:40.110878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b274df5-eeba-4c63-b4cd-9aa489cb9ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:40.111181 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b274df5-eeba-4c63-b4cd-9aa489cb9ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:40.130413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5b274df5-eeba-4c63-b4cd-9aa489cb9ee7 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:40.131712 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5b274df5-eeba-4c63-b4cd-9aa489cb9ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:40.191463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1422/2821] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:40.202224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:40.203999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:40.224467 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:40.264585 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae15a1-8288-4f6e-b151-2dc49fcde931 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:40.268736 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb63b3-0493-4a3c-b735-54541e5f7e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:40.269342 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb63b3-0493-4a3c-b735-54541e5f7e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:40.269613 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb63b3-0493-4a3c-b735-54541e5f7e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:40.269903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb63b3-0493-4a3c-b735-54541e5f7e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:40.279999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb63b3-0493-4a3c-b735-54541e5f7e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:40.281431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb63b3-0493-4a3c-b735-54541e5f7e24 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:40.292689 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1400/2822] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:40.713183 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b274df5-eeba-4c63-b4cd-9aa489cb9ee7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:40.714641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5b274df5-eeba-4c63-b4cd-9aa489cb9ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:40.728969 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b274df5-eeba-4c63-b4cd-9aa489cb9ee7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:40.734421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:40.735094 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:40.735416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:40.735689 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] PATH_INFO: `/v3/projects/8c6edf6ccbec416bafead82c1aced733` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:40.736822 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c6edf6ccbec416bafead82c1aced733)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:40.738737 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:40.858917 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c6edf6ccbec416bafead82c1aced733', 'request_id': 'req-2c018236-2921-4bd9-aa5c-3877408ea52f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.933457 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c6edf6ccbec416bafead82c1aced733', 'request_id': 'req-2c018236-2921-4bd9-aa5c-3877408ea52f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.941010 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c6edf6ccbec416bafead82c1aced733', 'request_id': 'req-2c018236-2921-4bd9-aa5c-3877408ea52f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.953318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] The token cache is being invalidate because project 8c6edf6ccbec416bafead82c1aced733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:36:40.953606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.954256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:40.955790 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1423/2823] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:36:40 2022] DELETE /identity/v3/projects/8c6edf6ccbec416bafead82c1aced733 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:40.960306 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb63b3-0493-4a3c-b735-54541e5f7e24 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:40.971222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb63b3-0493-4a3c-b735-54541e5f7e24 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:41.028405 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:41.029038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:41.029315 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:41.029588 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] PATH_INFO: `/v3/users/de1457f45c10415eb6d93949e40a9d9e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:41.030711 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1457f45c10415eb6d93949e40a9d9e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:41.032699 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:41.238819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1457f45c10415eb6d93949e40a9d9e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:41.239522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1457f45c10415eb6d93949e40a9d9e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:41.246422 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1457f45c10415eb6d93949e40a9d9e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:41.247084 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1457f45c10415eb6d93949e40a9d9e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:41.293506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-fd062aff-7c2c-437f-b808-12c59dc9f0c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1457f45c10415eb6d93949e40a9d9e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:41.312157 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1401/2824] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:36:40 2022] DELETE /identity/v3/users/de1457f45c10415eb6d93949e40a9d9e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:41.710780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:41.712313 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:41.725918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c018236-2921-4bd9-aa5c-3877408ea52f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:41.740433 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:41.789788 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:41.790543 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:41.790818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:41.791237 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] PATH_INFO: `/v3/projects/821b99792adc41e889e74feec32ecdbe` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:41.792497 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=821b99792adc41e889e74feec32ecdbe)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:41.795072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:41.896072 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '821b99792adc41e889e74feec32ecdbe', 'request_id': 'req-76aa461c-2720-4b33-b14b-fb72f38247f3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:41.988876 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '821b99792adc41e889e74feec32ecdbe', 'request_id': 'req-76aa461c-2720-4b33-b14b-fb72f38247f3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:41.994886 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '821b99792adc41e889e74feec32ecdbe', 'request_id': 'req-76aa461c-2720-4b33-b14b-fb72f38247f3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:42.012727 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] The token cache is being invalidate because project 821b99792adc41e889e74feec32ecdbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:36:42.013073 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:42.013968 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-76aa461c-2720-4b33-b14b-fb72f38247f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:42.015545 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1424/2825] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:36:41 2022] DELETE /identity/v3/projects/821b99792adc41e889e74feec32ecdbe => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:42.020459 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81932e43-3dae-4397-91c2-0316012d12fd None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.020940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81932e43-3dae-4397-91c2-0316012d12fd None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.021051 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81932e43-3dae-4397-91c2-0316012d12fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.041783 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-81932e43-3dae-4397-91c2-0316012d12fd None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:42.042989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-81932e43-3dae-4397-91c2-0316012d12fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:42.078737 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:42.103211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1402/2826] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:42.113284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b41e4-bc37-40e0-9a20-ebb58d509e50 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.113565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b41e4-bc37-40e0-9a20-ebb58d509e50 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.113836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b41e4-bc37-40e0-9a20-ebb58d509e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.134137 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-de1b41e4-bc37-40e0-9a20-ebb58d509e50 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:42.135504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-de1b41e4-bc37-40e0-9a20-ebb58d509e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:42.144900 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1425/2827] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:42.147615 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81932e43-3dae-4397-91c2-0316012d12fd None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:42.158378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81932e43-3dae-4397-91c2-0316012d12fd None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:42.163203 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:42.163840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.164117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.164412 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.165055 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:36:42.165358 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:42.167030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:42.180665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1403/2828] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:36:42.184256 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b41e4-bc37-40e0-9a20-ebb58d509e50 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:42.193327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b41e4-bc37-40e0-9a20-ebb58d509e50 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:42.197816 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddda13b-5343-45e1-9ea2-ca4f779e503c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:42.198399 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddda13b-5343-45e1-9ea2-ca4f779e503c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.198673 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddda13b-5343-45e1-9ea2-ca4f779e503c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.198950 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddda13b-5343-45e1-9ea2-ca4f779e503c admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.199575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddda13b-5343-45e1-9ea2-ca4f779e503c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:42.201454 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddda13b-5343-45e1-9ea2-ca4f779e503c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:42.317927 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1426/2829] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:36:42 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:42.322202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:42.332634 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5ccd9-9b92-4b31-9cda-1a2bc9b66217 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:42.390042 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601781d-15bb-4708-96bb-1f96004bcb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:42.390674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601781d-15bb-4708-96bb-1f96004bcb3f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.390970 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601781d-15bb-4708-96bb-1f96004bcb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.391386 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601781d-15bb-4708-96bb-1f96004bcb3f admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.392027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601781d-15bb-4708-96bb-1f96004bcb3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:42.393992 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601781d-15bb-4708-96bb-1f96004bcb3f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:42.515333 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1404/2830] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:36:42 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:36:42.519544 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddda13b-5343-45e1-9ea2-ca4f779e503c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:42.530324 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddda13b-5343-45e1-9ea2-ca4f779e503c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:42.534871 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed7021-71b1-4a4f-9df7-6d88008727a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:42.535496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed7021-71b1-4a4f-9df7-6d88008727a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.535776 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed7021-71b1-4a4f-9df7-6d88008727a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.536046 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed7021-71b1-4a4f-9df7-6d88008727a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.536688 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed7021-71b1-4a4f-9df7-6d88008727a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:42.538277 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed7021-71b1-4a4f-9df7-6d88008727a8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:42.548260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1427/2831] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:42.552249 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601781d-15bb-4708-96bb-1f96004bcb3f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:42.562514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601781d-15bb-4708-96bb-1f96004bcb3f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:42.567179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef906cf8-efd2-4bf5-8218-ef0184bfe092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:42.567762 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef906cf8-efd2-4bf5-8218-ef0184bfe092 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.568039 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef906cf8-efd2-4bf5-8218-ef0184bfe092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.568311 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef906cf8-efd2-4bf5-8218-ef0184bfe092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.568951 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef906cf8-efd2-4bf5-8218-ef0184bfe092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:42.570552 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef906cf8-efd2-4bf5-8218-ef0184bfe092 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:42.582248 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1405/2832] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:42.586223 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed7021-71b1-4a4f-9df7-6d88008727a8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:42.596850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed7021-71b1-4a4f-9df7-6d88008727a8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:42.601455 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc123914-7a4b-4194-b77c-81a8352110d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:42.602088 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc123914-7a4b-4194-b77c-81a8352110d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.602367 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc123914-7a4b-4194-b77c-81a8352110d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.602640 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc123914-7a4b-4194-b77c-81a8352110d4 admin admin] PATH_INFO: `/v3/projects/4246026aa1df429eb43818bc9f88cced/users/e6317b00304a4c71b2c61e165db843a5/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.617114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc123914-7a4b-4194-b77c-81a8352110d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4246026aa1df429eb43818bc9f88cced, user_id=e6317b00304a4c71b2c61e165db843a5, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:42.619796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc123914-7a4b-4194-b77c-81a8352110d4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:42.733780 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1428/2833] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:42 2022] PUT /identity/v3/projects/4246026aa1df429eb43818bc9f88cced/users/e6317b00304a4c71b2c61e165db843a5/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:42.744136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca05cf9-8c43-450f-b067-bec04ef87f22 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.744426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca05cf9-8c43-450f-b067-bec04ef87f22 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.744718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca05cf9-8c43-450f-b067-bec04ef87f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.773043 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-6ca05cf9-8c43-450f-b067-bec04ef87f22 None None] MFA Rules not processed for user `e6317b00304a4c71b2c61e165db843a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:42.774212 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-6ca05cf9-8c43-450f-b067-bec04ef87f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:42.837937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1406/2834] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:42.842988 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc123914-7a4b-4194-b77c-81a8352110d4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:42.892255 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:42.909381 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc123914-7a4b-4194-b77c-81a8352110d4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:42.914110 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cca75-3684-4a91-b111-0b6d9d20eecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:42.914703 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cca75-3684-4a91-b111-0b6d9d20eecc admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:42.915014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cca75-3684-4a91-b111-0b6d9d20eecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:42.915292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cca75-3684-4a91-b111-0b6d9d20eecc admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:42.915915 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cca75-3684-4a91-b111-0b6d9d20eecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:42.917843 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cca75-3684-4a91-b111-0b6d9d20eecc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:43.045111 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1429/2835] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:36:42 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:43.049319 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca05cf9-8c43-450f-b067-bec04ef87f22 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:43.064211 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca05cf9-8c43-450f-b067-bec04ef87f22 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:43.099102 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:43.125888 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeff6e-9a83-4997-952f-3cbaa83ba36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:43.126493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeff6e-9a83-4997-952f-3cbaa83ba36d admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.126776 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeff6e-9a83-4997-952f-3cbaa83ba36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.127082 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeff6e-9a83-4997-952f-3cbaa83ba36d admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:43.127704 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedeff6e-9a83-4997-952f-3cbaa83ba36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:43.129616 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedeff6e-9a83-4997-952f-3cbaa83ba36d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:43.247035 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1407/2836] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:36:43 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:43.250856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cca75-3684-4a91-b111-0b6d9d20eecc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:43.261174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cca75-3684-4a91-b111-0b6d9d20eecc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:43.266488 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b07d3-4059-4c02-984e-c71cc472f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:43.267114 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b07d3-4059-4c02-984e-c71cc472f3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.267393 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b07d3-4059-4c02-984e-c71cc472f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.267665 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b07d3-4059-4c02-984e-c71cc472f3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:43.268304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b07d3-4059-4c02-984e-c71cc472f3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:43.269992 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b07d3-4059-4c02-984e-c71cc472f3f3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:43.279882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1430/2837] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:43.283840 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeff6e-9a83-4997-952f-3cbaa83ba36d admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:43.294275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeff6e-9a83-4997-952f-3cbaa83ba36d admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:43.298990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1b55f-8aca-4357-9be8-26e255377b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:43.299530 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b55f-8aca-4357-9be8-26e255377b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.299809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b55f-8aca-4357-9be8-26e255377b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.300082 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b55f-8aca-4357-9be8-26e255377b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:43.300718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1b55f-8aca-4357-9be8-26e255377b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:43.302331 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1b55f-8aca-4357-9be8-26e255377b7c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:43.312632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1408/2838] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:43.316560 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b07d3-4059-4c02-984e-c71cc472f3f3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:43.326232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b07d3-4059-4c02-984e-c71cc472f3f3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:43.330999 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf88fd-f103-41fc-bd65-5f117591196c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:43.331627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf88fd-f103-41fc-bd65-5f117591196c admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.331903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf88fd-f103-41fc-bd65-5f117591196c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.332174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf88fd-f103-41fc-bd65-5f117591196c admin admin] PATH_INFO: `/v3/projects/41171f0dd9824c378f76e467117595d7/users/acf3b8b2ad8e434eb9713cd62a8e6ac3/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:43.346179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf88fd-f103-41fc-bd65-5f117591196c admin admin] RBAC: Authorizing `identity:create_grant(project_id=41171f0dd9824c378f76e467117595d7, user_id=acf3b8b2ad8e434eb9713cd62a8e6ac3, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:43.348882 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf88fd-f103-41fc-bd65-5f117591196c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:43.419208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1431/2839] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:43 2022] PUT /identity/v3/projects/41171f0dd9824c378f76e467117595d7/users/acf3b8b2ad8e434eb9713cd62a8e6ac3/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:36:43.429404 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1c6d-0572-4fca-9e2e-a2d2773c6766 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.429722 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1c6d-0572-4fca-9e2e-a2d2773c6766 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.429999 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1c6d-0572-4fca-9e2e-a2d2773c6766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:43.458145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-c46d1c6d-0572-4fca-9e2e-a2d2773c6766 None None] MFA Rules not processed for user `acf3b8b2ad8e434eb9713cd62a8e6ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:43.459398 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c46d1c6d-0572-4fca-9e2e-a2d2773c6766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:43.525231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1409/2840] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:43.529680 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf88fd-f103-41fc-bd65-5f117591196c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:43.590668 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf88fd-f103-41fc-bd65-5f117591196c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:43.595284 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9bf78-be1a-405a-a012-45a044a266b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:43.595878 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9bf78-be1a-405a-a012-45a044a266b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.596159 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9bf78-be1a-405a-a012-45a044a266b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.596430 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9bf78-be1a-405a-a012-45a044a266b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:43.597052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9bf78-be1a-405a-a012-45a044a266b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:43.598977 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9bf78-be1a-405a-a012-45a044a266b0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:43.676500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1432/2841] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:36:43 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:43.680898 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d1c6d-0572-4fca-9e2e-a2d2773c6766 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:43.691869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d1c6d-0572-4fca-9e2e-a2d2773c6766 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:43.751136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4be58b-f569-4c2d-8bf8-e5cfc71c0c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:43.751733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4be58b-f569-4c2d-8bf8-e5cfc71c0c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.752007 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4be58b-f569-4c2d-8bf8-e5cfc71c0c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.752270 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4be58b-f569-4c2d-8bf8-e5cfc71c0c25 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:43.752885 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4be58b-f569-4c2d-8bf8-e5cfc71c0c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:43.754837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4be58b-f569-4c2d-8bf8-e5cfc71c0c25 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:43.832370 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1410/2842] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:36:43 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:43.837031 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9bf78-be1a-405a-a012-45a044a266b0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:43.848351 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9bf78-be1a-405a-a012-45a044a266b0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:43.853299 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769b3d0-503a-452b-8f8e-0b4e4e0896f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:43.853895 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769b3d0-503a-452b-8f8e-0b4e4e0896f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.854174 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769b3d0-503a-452b-8f8e-0b4e4e0896f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.854444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769b3d0-503a-452b-8f8e-0b4e4e0896f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:43.855112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769b3d0-503a-452b-8f8e-0b4e4e0896f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:43.857325 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769b3d0-503a-452b-8f8e-0b4e4e0896f0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:43.867983 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1433/2843] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:43.874431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4be58b-f569-4c2d-8bf8-e5cfc71c0c25 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:43.885542 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4be58b-f569-4c2d-8bf8-e5cfc71c0c25 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:43.892123 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f48185-1c3d-46fa-a808-0fbdabc750f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:43.892771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f48185-1c3d-46fa-a808-0fbdabc750f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:43.893049 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f48185-1c3d-46fa-a808-0fbdabc750f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:43.893322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f48185-1c3d-46fa-a808-0fbdabc750f2 admin admin] PATH_INFO: `/v3/projects/304165f8838142eb89faa65c41524b7f/users/fd176238ece443d7b268b83003f73d5f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:44.030533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f48185-1c3d-46fa-a808-0fbdabc750f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304165f8838142eb89faa65c41524b7f, user_id=fd176238ece443d7b268b83003f73d5f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:44.033214 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f48185-1c3d-46fa-a808-0fbdabc750f2 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:44.117397 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1411/2844] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:43 2022] PUT /identity/v3/projects/304165f8838142eb89faa65c41524b7f/users/fd176238ece443d7b268b83003f73d5f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:44.121334 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769b3d0-503a-452b-8f8e-0b4e4e0896f0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:44.131763 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769b3d0-503a-452b-8f8e-0b4e4e0896f0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:44.188808 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924cda18-c269-44b5-ae9d-a3f46be5beb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:44.189416 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cda18-c269-44b5-ae9d-a3f46be5beb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:44.189694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cda18-c269-44b5-ae9d-a3f46be5beb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:44.189966 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cda18-c269-44b5-ae9d-a3f46be5beb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:44.190608 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cda18-c269-44b5-ae9d-a3f46be5beb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:44.192244 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cda18-c269-44b5-ae9d-a3f46be5beb5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:44.203267 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1434/2845] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:44.207030 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f48185-1c3d-46fa-a808-0fbdabc750f2 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:44.217718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f48185-1c3d-46fa-a808-0fbdabc750f2 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:44.222231 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:44.222860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:44.223162 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:44.223968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] PATH_INFO: `/v3/projects/304165f8838142eb89faa65c41524b7f/users/fd176238ece443d7b268b83003f73d5f/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:44.225964 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=304165f8838142eb89faa65c41524b7f, user_id=fd176238ece443d7b268b83003f73d5f, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:44.228602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:44.231767 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:44.239428 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1412/2846] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:44 2022] PUT /identity/v3/projects/304165f8838142eb89faa65c41524b7f/users/fd176238ece443d7b268b83003f73d5f/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:44.310217 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbe064-ee9f-494a-8808-5f26b7ee0798 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:44.310505 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbe064-ee9f-494a-8808-5f26b7ee0798 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:44.310783 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbe064-ee9f-494a-8808-5f26b7ee0798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:44.318062 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:44.341412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dbfbe064-ee9f-494a-8808-5f26b7ee0798 None None] MFA Rules not processed for user `fd176238ece443d7b268b83003f73d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:44.342629 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dbfbe064-ee9f-494a-8808-5f26b7ee0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:44.410682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1435/2847] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:44.421771 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:44.423448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:44.483299 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c8297-42bf-40fc-87e6-69f771a0b6bc admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:44.487070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4a8e2-a45d-49c1-8709-a9fb9b55bc80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:44.487678 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a8e2-a45d-49c1-8709-a9fb9b55bc80 service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:44.487953 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a8e2-a45d-49c1-8709-a9fb9b55bc80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:44.488222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a8e2-a45d-49c1-8709-a9fb9b55bc80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:44.499002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4a8e2-a45d-49c1-8709-a9fb9b55bc80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:44.500517 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4a8e2-a45d-49c1-8709-a9fb9b55bc80 service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:44.511136 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1413/2848] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:44.660547 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbe064-ee9f-494a-8808-5f26b7ee0798 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:44.662091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dbfbe064-ee9f-494a-8808-5f26b7ee0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:44.676562 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbe064-ee9f-494a-8808-5f26b7ee0798 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:44.680694 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:44.681883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:44.681883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:44.682185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:44.761164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:44.764190 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:44.775052 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1436/2849] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 08:36:51.297379 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c611b83-61c3-469b-bae5-27bbbb5bf3a7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:51.297915 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c611b83-61c3-469b-bae5-27bbbb5bf3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:51.298022 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c611b83-61c3-469b-bae5-27bbbb5bf3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:51.317351 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-0c611b83-61c3-469b-bae5-27bbbb5bf3a7 None None] MFA Rules not processed for user `db8b5a8edcc34d32a3e9fad54d33cbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:51.318570 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-0c611b83-61c3-469b-bae5-27bbbb5bf3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:51.374382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1414/2850] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:51.695548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:51.696940 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:51.755418 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0e8e1-ed1f-4158-9f89-66bfc02c4896 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:51.759014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab102c-45f8-4596-89aa-035b2cc85518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:51.759616 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab102c-45f8-4596-89aa-035b2cc85518 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:51.759891 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab102c-45f8-4596-89aa-035b2cc85518 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:51.760158 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab102c-45f8-4596-89aa-035b2cc85518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:51.761975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eeab102c-45f8-4596-89aa-035b2cc85518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:51.772436 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab102c-45f8-4596-89aa-035b2cc85518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:51.773866 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab102c-45f8-4596-89aa-035b2cc85518 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:51.782631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1437/2851] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:51.985378 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c611b83-61c3-469b-bae5-27bbbb5bf3a7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:52.054734 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c611b83-61c3-469b-bae5-27bbbb5bf3a7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:52.060805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:52.061441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:52.061715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:52.061981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] PATH_INFO: `/v3/users/e6317b00304a4c71b2c61e165db843a5` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:52.063144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6317b00304a4c71b2c61e165db843a5)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:52.065094 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:52.295269 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:52.296732 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6317b00304a4c71b2c61e165db843a5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:52.297382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6317b00304a4c71b2c61e165db843a5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:52.304209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6317b00304a4c71b2c61e165db843a5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:52.304837 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6317b00304a4c71b2c61e165db843a5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:52.425594 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6317b00304a4c71b2c61e165db843a5'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:52.443448 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1415/2852] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:36:51 2022] DELETE /identity/v3/users/e6317b00304a4c71b2c61e165db843a5 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:36:52.448053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d41b-3ef9-4139-ae6c-17345e196870 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:52.448343 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d41b-3ef9-4139-ae6c-17345e196870 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:52.448618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d41b-3ef9-4139-ae6c-17345e196870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:52.469163 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-74b1d41b-3ef9-4139-ae6c-17345e196870 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:52.470364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-74b1d41b-3ef9-4139-ae6c-17345e196870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:52.494833 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:52.530814 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1438/2853] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:52.540654 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:52.542077 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:52.601132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8c5bb-ce48-4c79-8f72-72fbf88c4a6e admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:52.604662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83699a72-52d8-411a-ae8d-60630868f035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:52.605293 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83699a72-52d8-411a-ae8d-60630868f035 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:52.605532 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83699a72-52d8-411a-ae8d-60630868f035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:52.605865 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83699a72-52d8-411a-ae8d-60630868f035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:52.614895 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83699a72-52d8-411a-ae8d-60630868f035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:52.616340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83699a72-52d8-411a-ae8d-60630868f035 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:52.626224 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1416/2854] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:53.041939 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1d41b-3ef9-4139-ae6c-17345e196870 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:53.044039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-74b1d41b-3ef9-4139-ae6c-17345e196870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:53.056611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1d41b-3ef9-4139-ae6c-17345e196870 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:53.061934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:53.062528 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:53.062809 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:53.063116 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] PATH_INFO: `/v3/projects/4246026aa1df429eb43818bc9f88cced` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:53.064380 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4246026aa1df429eb43818bc9f88cced)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:53.066300 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:53.169274 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4246026aa1df429eb43818bc9f88cced', 'request_id': 'req-56c13575-6727-40c5-ab7f-b89c605cf59c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.219437 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4246026aa1df429eb43818bc9f88cced', 'request_id': 'req-56c13575-6727-40c5-ab7f-b89c605cf59c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.225730 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4246026aa1df429eb43818bc9f88cced', 'request_id': 'req-56c13575-6727-40c5-ab7f-b89c605cf59c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.237565 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] The token cache is being invalidate because project 4246026aa1df429eb43818bc9f88cced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:36:53.237888 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.238579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.240164 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1439/2855] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:36:53 2022] DELETE /identity/v3/projects/4246026aa1df429eb43818bc9f88cced => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:53.244111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83699a72-52d8-411a-ae8d-60630868f035 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:53.254484 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83699a72-52d8-411a-ae8d-60630868f035 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:53.308308 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:53.309566 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:53.309824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:53.310098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] PATH_INFO: `/v3/users/acf3b8b2ad8e434eb9713cd62a8e6ac3` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:53.311273 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf3b8b2ad8e434eb9713cd62a8e6ac3)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:53.313183 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:53.523272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf3b8b2ad8e434eb9713cd62a8e6ac3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.523903 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf3b8b2ad8e434eb9713cd62a8e6ac3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.531385 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf3b8b2ad8e434eb9713cd62a8e6ac3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.532044 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf3b8b2ad8e434eb9713cd62a8e6ac3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.602779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf3b8b2ad8e434eb9713cd62a8e6ac3'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:53.608764 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:53.622813 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1417/2856] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:36:53 2022] DELETE /identity/v3/users/acf3b8b2ad8e434eb9713cd62a8e6ac3 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:54.041863 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:54.043353 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:54.056327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c13575-6727-40c5-ab7f-b89c605cf59c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:54.114340 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:54.114996 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:54.115281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:54.115555 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] PATH_INFO: `/v3/projects/41171f0dd9824c378f76e467117595d7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:54.116682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41171f0dd9824c378f76e467117595d7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:54.118612 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:54.223802 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41171f0dd9824c378f76e467117595d7', 'request_id': 'req-933bcff9-9828-4bc7-9bae-0020f7709608'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.227503 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:54.354143 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41171f0dd9824c378f76e467117595d7', 'request_id': 'req-933bcff9-9828-4bc7-9bae-0020f7709608'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.361356 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41171f0dd9824c378f76e467117595d7', 'request_id': 'req-933bcff9-9828-4bc7-9bae-0020f7709608'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.375813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] The token cache is being invalidate because project 41171f0dd9824c378f76e467117595d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:36:54.384091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.384091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.384091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1440/2857] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:36:54 2022] DELETE /identity/v3/projects/41171f0dd9824c378f76e467117595d7 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:54.384972 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:54.387132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:54.402987 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede935f5-d6c6-4dac-9a38-57247dcc1f88 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:54.460226 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:54.460860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:54.461140 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:54.461414 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] PATH_INFO: `/v3/users/fd176238ece443d7b268b83003f73d5f` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:54.462539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd176238ece443d7b268b83003f73d5f)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:54.464591 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:54.633418 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd176238ece443d7b268b83003f73d5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.633990 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd176238ece443d7b268b83003f73d5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.640996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd176238ece443d7b268b83003f73d5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.641648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd176238ece443d7b268b83003f73d5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.699481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-07f7c96a-ae39-4b01-8282-c5fecc6f78b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd176238ece443d7b268b83003f73d5f'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:54.718483 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1418/2858] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:36:54 2022] DELETE /identity/v3/users/fd176238ece443d7b268b83003f73d5f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:55.175793 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:55.176252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:55.190828 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bcff9-9828-4bc7-9bae-0020f7709608 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:55.248972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:55.249620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:55.249899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:55.250171 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] PATH_INFO: `/v3/projects/304165f8838142eb89faa65c41524b7f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:55.251409 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=304165f8838142eb89faa65c41524b7f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:55.253318 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:55.366090 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304165f8838142eb89faa65c41524b7f', 'request_id': 'req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:55.440717 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304165f8838142eb89faa65c41524b7f', 'request_id': 'req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:55.446623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304165f8838142eb89faa65c41524b7f', 'request_id': 'req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:55.458502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] The token cache is being invalidate because project 304165f8838142eb89faa65c41524b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:36:55.458792 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:55.459463 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8feac87d-8dfd-4c4a-a51e-4733aab6bbbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:36:55.460945 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1441/2859] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:36:55 2022] DELETE /identity/v3/projects/304165f8838142eb89faa65c41524b7f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:55.467425 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae56a9-e32d-4fa1-9193-b54ddc0b8b30 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:55.467955 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae56a9-e32d-4fa1-9193-b54ddc0b8b30 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:55.468121 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae56a9-e32d-4fa1-9193-b54ddc0b8b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:55.487559 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-acae56a9-e32d-4fa1-9193-b54ddc0b8b30 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:55.495315 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-acae56a9-e32d-4fa1-9193-b54ddc0b8b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:55.546821 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1419/2860] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:55.555123 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de376f-9cdb-4684-9d2b-60ee892a514a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:55.555413 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de376f-9cdb-4684-9d2b-60ee892a514a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:55.555695 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de376f-9cdb-4684-9d2b-60ee892a514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:55.575489 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-57de376f-9cdb-4684-9d2b-60ee892a514a None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:55.576699 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-57de376f-9cdb-4684-9d2b-60ee892a514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:55.584799 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1442/2861] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:55.588969 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae56a9-e32d-4fa1-9193-b54ddc0b8b30 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:55.595869 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:55.599617 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae56a9-e32d-4fa1-9193-b54ddc0b8b30 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:55.604383 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:55.604973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:55.605247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:55.605515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:55.606210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:36:55.606507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:55.608179 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:55.621233 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1420/2862] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:36:55.625047 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de376f-9cdb-4684-9d2b-60ee892a514a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:55.634558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de376f-9cdb-4684-9d2b-60ee892a514a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:55.639923 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682162cd-b7ce-4309-ac57-58f7457d72ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:55.640520 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682162cd-b7ce-4309-ac57-58f7457d72ad admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:55.640796 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682162cd-b7ce-4309-ac57-58f7457d72ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:55.641063 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682162cd-b7ce-4309-ac57-58f7457d72ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:55.641685 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682162cd-b7ce-4309-ac57-58f7457d72ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:55.643617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682162cd-b7ce-4309-ac57-58f7457d72ad admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:55.769926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1443/2863] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:36:55 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:55.774296 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:55.785718 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64164f9-b580-4408-a8a5-73d4415231a4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:55.844819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6e329-93b3-47f8-ab6d-27ad568e424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:55.845426 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6e329-93b3-47f8-ab6d-27ad568e424f admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:55.845707 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6e329-93b3-47f8-ab6d-27ad568e424f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:55.845977 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6e329-93b3-47f8-ab6d-27ad568e424f admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:55.846642 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6e329-93b3-47f8-ab6d-27ad568e424f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:55.848593 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6e329-93b3-47f8-ab6d-27ad568e424f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:55.963114 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1421/2864] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:36:55 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 08:36:55.967368 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682162cd-b7ce-4309-ac57-58f7457d72ad admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:55.977933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682162cd-b7ce-4309-ac57-58f7457d72ad admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:55.982773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b771bb2-9032-4df8-9c63-e049294be7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:55.983517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b771bb2-9032-4df8-9c63-e049294be7aa admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:55.983702 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b771bb2-9032-4df8-9c63-e049294be7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:55.984205 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b771bb2-9032-4df8-9c63-e049294be7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:55.984777 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b771bb2-9032-4df8-9c63-e049294be7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:55.986451 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b771bb2-9032-4df8-9c63-e049294be7aa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:55.999202 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1444/2865] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:56.003272 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6e329-93b3-47f8-ab6d-27ad568e424f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:56.014048 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6e329-93b3-47f8-ab6d-27ad568e424f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:56.018631 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c429688-46f4-4293-b9a4-e2148bf36313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:56.019260 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429688-46f4-4293-b9a4-e2148bf36313 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:56.022855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429688-46f4-4293-b9a4-e2148bf36313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:56.022855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429688-46f4-4293-b9a4-e2148bf36313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:56.022855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c429688-46f4-4293-b9a4-e2148bf36313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:56.022855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c429688-46f4-4293-b9a4-e2148bf36313 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:56.032618 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1422/2866] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:56.036696 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b771bb2-9032-4df8-9c63-e049294be7aa admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:56.047314 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b771bb2-9032-4df8-9c63-e049294be7aa admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:56.051918 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:56.052559 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:56.052838 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:56.053112 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] PATH_INFO: `/v3/projects/ed688a27a2914c9f9c3ac33e5e380500/users/4ce7c9048a1b439d8b0f039f88692894/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:56.068934 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed688a27a2914c9f9c3ac33e5e380500, user_id=4ce7c9048a1b439d8b0f039f88692894, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:56.071686 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:56.162185 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1445/2867] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:56 2022] PUT /identity/v3/projects/ed688a27a2914c9f9c3ac33e5e380500/users/4ce7c9048a1b439d8b0f039f88692894/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:56.167266 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d84c0-4f92-4162-b3ad-91604e30d4c9 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:56.167548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d84c0-4f92-4162-b3ad-91604e30d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:56.167824 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d84c0-4f92-4162-b3ad-91604e30d4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:56.187665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-091d84c0-4f92-4162-b3ad-91604e30d4c9 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:56.189145 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-091d84c0-4f92-4162-b3ad-91604e30d4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:56.247962 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1423/2868] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:56.257476 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:56.259232 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:56.270129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:56.316182 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005c708-4f36-4a2f-9ba0-30fbb8d21096 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:56.319903 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f62060-4071-4d3e-a37a-828fabdc1124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:56.326537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62060-4071-4d3e-a37a-828fabdc1124 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:56.326537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62060-4071-4d3e-a37a-828fabdc1124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:56.326537 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62060-4071-4d3e-a37a-828fabdc1124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:56.330470 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f62060-4071-4d3e-a37a-828fabdc1124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:56.331953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f62060-4071-4d3e-a37a-828fabdc1124 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:56.341849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1446/2869] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:36:57.501071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb375-5296-4708-ae91-6589d20dfcc7 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:57.501635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb375-5296-4708-ae91-6589d20dfcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:57.501635 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb375-5296-4708-ae91-6589d20dfcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:57.542991 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-f4afb375-5296-4708-ae91-6589d20dfcc7 None None] MFA Rules not processed for user `4ce7c9048a1b439d8b0f039f88692894`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:57.544877 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f4afb375-5296-4708-ae91-6589d20dfcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:57.611362 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:57.614867 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1424/2870] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:57.619892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f62060-4071-4d3e-a37a-828fabdc1124 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:57.635755 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f62060-4071-4d3e-a37a-828fabdc1124 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:57.638521 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4470f6-235c-4b6c-8978-f63db56c8ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:57.639177 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4470f6-235c-4b6c-8978-f63db56c8ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:57.639456 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4470f6-235c-4b6c-8978-f63db56c8ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:57.639726 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4470f6-235c-4b6c-8978-f63db56c8ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:57.640359 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4470f6-235c-4b6c-8978-f63db56c8ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:57.642333 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4470f6-235c-4b6c-8978-f63db56c8ef8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:57.741023 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1447/2871] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:36:57 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:57.745180 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afb375-5296-4708-ae91-6589d20dfcc7 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:57.755981 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afb375-5296-4708-ae91-6589d20dfcc7 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:57.814281 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62049ee8-4758-431b-a4e6-edafbe8aa5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:57.814989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62049ee8-4758-431b-a4e6-edafbe8aa5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:57.815279 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62049ee8-4758-431b-a4e6-edafbe8aa5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:57.815549 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62049ee8-4758-431b-a4e6-edafbe8aa5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:57.816197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62049ee8-4758-431b-a4e6-edafbe8aa5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:57.818151 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62049ee8-4758-431b-a4e6-edafbe8aa5a8 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:57.924021 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1425/2872] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:36:57 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:36:57.927930 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4470f6-235c-4b6c-8978-f63db56c8ef8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:57.939039 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4470f6-235c-4b6c-8978-f63db56c8ef8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:57.946846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9405267-ee7f-4842-85ce-21ef786f2de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:57.947850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9405267-ee7f-4842-85ce-21ef786f2de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:57.948297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9405267-ee7f-4842-85ce-21ef786f2de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:57.948732 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9405267-ee7f-4842-85ce-21ef786f2de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:57.949821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9405267-ee7f-4842-85ce-21ef786f2de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:57.952639 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9405267-ee7f-4842-85ce-21ef786f2de1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:57.972296 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1448/2873] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:57.976580 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62049ee8-4758-431b-a4e6-edafbe8aa5a8 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:57.989316 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62049ee8-4758-431b-a4e6-edafbe8aa5a8 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:57.994210 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c813-e945-4972-9d80-2961157b4d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:57.994878 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c813-e945-4972-9d80-2961157b4d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:57.995243 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c813-e945-4972-9d80-2961157b4d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:57.995500 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c813-e945-4972-9d80-2961157b4d3c admin admin] PATH_INFO: `/v3/projects/02904eb8c0a24fa59f8de13083af7e7a/users/7b6e1b539f354812b042a38a7944639e/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:58.011798 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93c813-e945-4972-9d80-2961157b4d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=02904eb8c0a24fa59f8de13083af7e7a, user_id=7b6e1b539f354812b042a38a7944639e, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:58.014435 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93c813-e945-4972-9d80-2961157b4d3c admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:58.122768 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1426/2874] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:57 2022] PUT /identity/v3/projects/02904eb8c0a24fa59f8de13083af7e7a/users/7b6e1b539f354812b042a38a7944639e/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:36:58.126894 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9405267-ee7f-4842-85ce-21ef786f2de1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:58.137234 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9405267-ee7f-4842-85ce-21ef786f2de1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:58.193858 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df950f9-33d7-4d55-9324-d753d382def8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:58.194406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df950f9-33d7-4d55-9324-d753d382def8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:58.194683 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df950f9-33d7-4d55-9324-d753d382def8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:58.194975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df950f9-33d7-4d55-9324-d753d382def8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:58.195617 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df950f9-33d7-4d55-9324-d753d382def8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:58.197211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df950f9-33d7-4d55-9324-d753d382def8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:58.207611 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1449/2875] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:36:58.211886 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c813-e945-4972-9d80-2961157b4d3c admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:58.222706 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c813-e945-4972-9d80-2961157b4d3c admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:58.227482 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:58.228132 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:58.228416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:58.228715 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] PATH_INFO: `/v3/projects/02904eb8c0a24fa59f8de13083af7e7a/users/7b6e1b539f354812b042a38a7944639e/roles/63edd667e7714fc08d3f32f6682c6369` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:58.230649 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02904eb8c0a24fa59f8de13083af7e7a, user_id=7b6e1b539f354812b042a38a7944639e, role_id=63edd667e7714fc08d3f32f6682c6369)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:58.233882 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:58.244158 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1427/2876] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:36:58 2022] PUT /identity/v3/projects/02904eb8c0a24fa59f8de13083af7e7a/users/7b6e1b539f354812b042a38a7944639e/roles/63edd667e7714fc08d3f32f6682c6369 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:36:59.578330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699c3e0-c5f3-4283-b456-a87edb529b9e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:59.579193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699c3e0-c5f3-4283-b456-a87edb529b9e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:59.579193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699c3e0-c5f3-4283-b456-a87edb529b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:59.612744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4699c3e0-c5f3-4283-b456-a87edb529b9e None None] MFA Rules not processed for user `7b6e1b539f354812b042a38a7944639e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:36:59.612744 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4699c3e0-c5f3-4283-b456-a87edb529b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:59.684925 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:36:59.689773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1450/2877] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:36:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:36:59.702071 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:36:59.704940 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:36:59.774640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee7b5f-d811-4b2f-ad2c-d2dbe57674a4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:36:59.779602 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:36:59.780247 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:36:59.780583 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:36:59.780862 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:36:59.846623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:36:59.848197 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:36:59.859171 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1428/2878] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:36:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:37:02.640449 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699c3e0-c5f3-4283-b456-a87edb529b9e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:02.641864 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4699c3e0-c5f3-4283-b456-a87edb529b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:02.654382 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699c3e0-c5f3-4283-b456-a87edb529b9e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:02.658211 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:02.658810 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:02.659122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:02.659398 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:02.669208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:02.670642 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:02.680421 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1451/2879] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:37:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:37:09.853623 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:12.676049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:16.940848 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:16.942344 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:16.999117 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a99366-1075-481e-bbe3-bb1ba019198d service cinder] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:17.002805 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:17.003441 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:17.003723 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:17.003996 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:17.005691 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:17.068205 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:17.070809 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:17.080015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1429/2880] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:37:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:37:20.432042 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:20.433623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:20.508198 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc07350-6735-4c84-9fca-4cb8df7192f5 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:20.513953 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:20.514553 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:20.514831 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:20.515141 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:20.517049 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:20.528836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:20.530406 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:20.540564 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1452/2881] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:37:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:37:21.345563 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:21.414705 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588ee37-969e-4fbb-b08c-fda948d7cd69 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:21.420274 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:21.420968 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:21.421250 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:21.421553 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] PATH_INFO: `/v3/users/4ce7c9048a1b439d8b0f039f88692894` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:21.422819 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce7c9048a1b439d8b0f039f88692894)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:21.424973 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:21.755407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce7c9048a1b439d8b0f039f88692894'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:21.756146 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce7c9048a1b439d8b0f039f88692894'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:21.764946 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:21.768870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce7c9048a1b439d8b0f039f88692894'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:21.769751 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce7c9048a1b439d8b0f039f88692894'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:21.930823 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce7c9048a1b439d8b0f039f88692894'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:21.952411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1430/2882] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:37:21 2022] DELETE /identity/v3/users/4ce7c9048a1b439d8b0f039f88692894 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:37:22.285242 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:22.298785 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f7510-a9a9-42bc-b371-e69f77a28dea service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:22.375308 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:22.376344 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:22.376856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:22.377350 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] PATH_INFO: `/v3/projects/ed688a27a2914c9f9c3ac33e5e380500` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:22.379037 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed688a27a2914c9f9c3ac33e5e380500)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:22.382178 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:22.555893 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed688a27a2914c9f9c3ac33e5e380500', 'request_id': 'req-1b9706db-9873-4c88-87d2-8a522c7eaa89'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:22.659369 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed688a27a2914c9f9c3ac33e5e380500', 'request_id': 'req-1b9706db-9873-4c88-87d2-8a522c7eaa89'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:22.667517 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed688a27a2914c9f9c3ac33e5e380500', 'request_id': 'req-1b9706db-9873-4c88-87d2-8a522c7eaa89'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:22.680797 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] The token cache is being invalidate because project ed688a27a2914c9f9c3ac33e5e380500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:37:22.681084 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:22.681824 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:22.683496 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1453/2883] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:37:22 2022] DELETE /identity/v3/projects/ed688a27a2914c9f9c3ac33e5e380500 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:37:22.687986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:22.689604 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:22.702860 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9038a4e-47fb-4460-a031-4fb47e9e49df admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:22.764601 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:22.765236 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:22.765515 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:22.765779 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] PATH_INFO: `/v3/users/7b6e1b539f354812b042a38a7944639e` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:22.767111 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6e1b539f354812b042a38a7944639e)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:22.768933 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:23.039307 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6e1b539f354812b042a38a7944639e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.040066 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6e1b539f354812b042a38a7944639e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.050144 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6e1b539f354812b042a38a7944639e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.050795 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6e1b539f354812b042a38a7944639e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.172661 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-86b1e3d9-f694-44aa-9caa-fb029becd08d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6e1b539f354812b042a38a7944639e'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.191690 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1431/2884] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:37:22 2022] DELETE /identity/v3/users/7b6e1b539f354812b042a38a7944639e => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:37:23.513294 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:23.514766 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:23.532862 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9706db-9873-4c88-87d2-8a522c7eaa89 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:23.554281 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:23.601830 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:23.602465 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:23.602741 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:23.603068 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] PATH_INFO: `/v3/projects/02904eb8c0a24fa59f8de13083af7e7a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:23.604260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02904eb8c0a24fa59f8de13083af7e7a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:23.606236 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:23.742997 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02904eb8c0a24fa59f8de13083af7e7a', 'request_id': 'req-389e83a8-ccde-4c7b-9af2-65edf58bc779'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.869511 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02904eb8c0a24fa59f8de13083af7e7a', 'request_id': 'req-389e83a8-ccde-4c7b-9af2-65edf58bc779'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.877714 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02904eb8c0a24fa59f8de13083af7e7a', 'request_id': 'req-389e83a8-ccde-4c7b-9af2-65edf58bc779'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.891618 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] The token cache is being invalidate because project 02904eb8c0a24fa59f8de13083af7e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:37:23.891946 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.892720 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-389e83a8-ccde-4c7b-9af2-65edf58bc779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:23.894765 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1454/2885] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:37:23 2022] DELETE /identity/v3/projects/02904eb8c0a24fa59f8de13083af7e7a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:37:23.901539 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b49e48-8efd-4917-86b6-60eb94db7f57 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:23.902009 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b49e48-8efd-4917-86b6-60eb94db7f57 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:23.904810 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b49e48-8efd-4917-86b6-60eb94db7f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:23.924582 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-68b49e48-8efd-4917-86b6-60eb94db7f57 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:37:23.925892 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-68b49e48-8efd-4917-86b6-60eb94db7f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:23.971223 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:23.986760 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1432/2886] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:37:23.994561 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af90c4-2ef3-4173-a209-eed33c08f9cd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:23.994845 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af90c4-2ef3-4173-a209-eed33c08f9cd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:23.995153 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af90c4-2ef3-4173-a209-eed33c08f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.015975 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-80af90c4-2ef3-4173-a209-eed33c08f9cd None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:37:24.017167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-80af90c4-2ef3-4173-a209-eed33c08f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:24.025601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1455/2887] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:37:24.030028 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b49e48-8efd-4917-86b6-60eb94db7f57 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:24.040120 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b49e48-8efd-4917-86b6-60eb94db7f57 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:24.046147 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:24.046741 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.047050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.047324 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.047995 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:37:24.048298 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:24.049891 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:24.062357 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1433/2888] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:37:24.066818 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af90c4-2ef3-4173-a209-eed33c08f9cd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:24.076161 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af90c4-2ef3-4173-a209-eed33c08f9cd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:24.081266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556f855-92e6-4ca1-8bc6-c53ce907af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:24.081849 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f855-92e6-4ca1-8bc6-c53ce907af5e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.082122 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f855-92e6-4ca1-8bc6-c53ce907af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.082412 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556f855-92e6-4ca1-8bc6-c53ce907af5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.083030 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556f855-92e6-4ca1-8bc6-c53ce907af5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:24.085091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556f855-92e6-4ca1-8bc6-c53ce907af5e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:24.233760 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1456/2889] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:37:24 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:37:24.237948 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:24.248187 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcfd8c-fc4a-4a9c-9667-57cede4ad560 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:24.307246 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90d6cf-a9da-4232-ab55-9006ccec83b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:24.307827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d6cf-a9da-4232-ab55-9006ccec83b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.308100 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d6cf-a9da-4232-ab55-9006ccec83b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.308364 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d6cf-a9da-4232-ab55-9006ccec83b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.308989 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90d6cf-a9da-4232-ab55-9006ccec83b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:24.310997 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90d6cf-a9da-4232-ab55-9006ccec83b5 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:24.444178 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1434/2890] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:37:24 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:37:24.448209 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556f855-92e6-4ca1-8bc6-c53ce907af5e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:24.458662 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556f855-92e6-4ca1-8bc6-c53ce907af5e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:24.463394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1667905-9945-4a63-ae49-43dbe08b293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:24.463984 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1667905-9945-4a63-ae49-43dbe08b293e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.464257 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1667905-9945-4a63-ae49-43dbe08b293e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.464522 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1667905-9945-4a63-ae49-43dbe08b293e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.465648 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1667905-9945-4a63-ae49-43dbe08b293e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:24.467364 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1667905-9945-4a63-ae49-43dbe08b293e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:24.477926 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1457/2891] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:37:24.483865 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90d6cf-a9da-4232-ab55-9006ccec83b5 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:24.494371 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90d6cf-a9da-4232-ab55-9006ccec83b5 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:24.499072 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b78c72-a93b-4857-8a8d-f8cb94f9fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:24.499733 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b78c72-a93b-4857-8a8d-f8cb94f9fb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.500012 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b78c72-a93b-4857-8a8d-f8cb94f9fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.500280 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b78c72-a93b-4857-8a8d-f8cb94f9fb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.500919 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b78c72-a93b-4857-8a8d-f8cb94f9fb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:24.502507 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b78c72-a93b-4857-8a8d-f8cb94f9fb03 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:24.513961 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1435/2892] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:37:24.518643 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1667905-9945-4a63-ae49-43dbe08b293e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:24.529500 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1667905-9945-4a63-ae49-43dbe08b293e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:24.534126 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:24.534798 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.535187 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.535466 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] PATH_INFO: `/v3/projects/4a6d412070824db9a80ae74f511af9ed/users/f203b815c5d147b58b6b37a2704a7ae9/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.553279 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6d412070824db9a80ae74f511af9ed, user_id=f203b815c5d147b58b6b37a2704a7ae9, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:24.555985 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:24.640233 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1458/2893] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:37:24 2022] PUT /identity/v3/projects/4a6d412070824db9a80ae74f511af9ed/users/f203b815c5d147b58b6b37a2704a7ae9/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:37:24.652209 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fb336-6ed0-4e25-b9ee-40918995f0e3 None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.652522 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fb336-6ed0-4e25-b9ee-40918995f0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.653322 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fb336-6ed0-4e25-b9ee-40918995f0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.684050 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-147fb336-6ed0-4e25-b9ee-40918995f0e3 None None] MFA Rules not processed for user `f203b815c5d147b58b6b37a2704a7ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:37:24.685251 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-147fb336-6ed0-4e25-b9ee-40918995f0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:24.751033 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1436/2894] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:37:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 08:37:24.764627 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:24.766502 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:24.830289 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:24.836681 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81318a13-934d-4bcd-93c3-1216152d2958 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:24.841195 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a343b-792e-4862-9dce-c43bb4095b64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:24.845749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a343b-792e-4862-9dce-c43bb4095b64 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.845749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a343b-792e-4862-9dce-c43bb4095b64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.845749 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a343b-792e-4862-9dce-c43bb4095b64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.852534 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a343b-792e-4862-9dce-c43bb4095b64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:24.854003 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a343b-792e-4862-9dce-c43bb4095b64 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:24.863747 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1459/2895] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:37:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:37:24.891275 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fb336-6ed0-4e25-b9ee-40918995f0e3 None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:24.942005 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:24.961327 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fb336-6ed0-4e25-b9ee-40918995f0e3 None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:24.966329 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:24.966982 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:24.967263 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:24.967533 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] PATH_INFO: `/v3/users/f203b815c5d147b58b6b37a2704a7ae9` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:24.969447 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f203b815c5d147b58b6b37a2704a7ae9)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:24.971431 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:25.435870 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f203b815c5d147b58b6b37a2704a7ae9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:25.436540 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f203b815c5d147b58b6b37a2704a7ae9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:25.443650 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f203b815c5d147b58b6b37a2704a7ae9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:25.444315 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f203b815c5d147b58b6b37a2704a7ae9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:25.626481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f203b815c5d147b58b6b37a2704a7ae9'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:25.645833 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1437/2896] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:37:24 2022] DELETE /identity/v3/users/f203b815c5d147b58b6b37a2704a7ae9 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:37:25.650354 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5ce34-6448-4d1d-8cc1-c37c2fd69afe None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:25.650914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5ce34-6448-4d1d-8cc1-c37c2fd69afe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:25.650914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5ce34-6448-4d1d-8cc1-c37c2fd69afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:25.672179 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b9a5ce34-6448-4d1d-8cc1-c37c2fd69afe None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:37:25.673892 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b9a5ce34-6448-4d1d-8cc1-c37c2fd69afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:25.740027 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1460/2897] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:37:25.750015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:25.751587 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:25.823074 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fc9d8-d26c-40f4-9b2b-508b0d4de9f9 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:25.828411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ed328-4dc1-4423-9270-d0f607e26213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:25.829325 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ed328-4dc1-4423-9270-d0f607e26213 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:25.829994 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ed328-4dc1-4423-9270-d0f607e26213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:25.830396 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ed328-4dc1-4423-9270-d0f607e26213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:25.845200 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ed328-4dc1-4423-9270-d0f607e26213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:25.846659 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ed328-4dc1-4423-9270-d0f607e26213 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:25.857684 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1438/2898] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:37:26.381051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5ce34-6448-4d1d-8cc1-c37c2fd69afe None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:26.382581 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b9a5ce34-6448-4d1d-8cc1-c37c2fd69afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:26.402601 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5ce34-6448-4d1d-8cc1-c37c2fd69afe None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:26.407883 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:26.408548 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:26.408821 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:26.409091 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] PATH_INFO: `/v3/projects/4a6d412070824db9a80ae74f511af9ed` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:26.410667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6d412070824db9a80ae74f511af9ed)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:26.412582 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:26.570868 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6d412070824db9a80ae74f511af9ed', 'request_id': 'req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:26.706836 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6d412070824db9a80ae74f511af9ed', 'request_id': 'req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:26.714016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6d412070824db9a80ae74f511af9ed', 'request_id': 'req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:26.725813 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] The token cache is being invalidate because project 4a6d412070824db9a80ae74f511af9ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:37:26.726140 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:26.726773 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd3ebcb4-1798-4fd2-8767-4abdb26fa50b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:26.728303 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1461/2899] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:37:26 2022] DELETE /identity/v3/projects/4a6d412070824db9a80ae74f511af9ed => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:37:26.733780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa57894-aae4-43ca-84e3-60a7b42da7ab None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:26.733780 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa57894-aae4-43ca-84e3-60a7b42da7ab None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:26.734382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa57894-aae4-43ca-84e3-60a7b42da7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:26.754070 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-efa57894-aae4-43ca-84e3-60a7b42da7ab None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:37:26.755346 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-efa57894-aae4-43ca-84e3-60a7b42da7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:26.818468 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1439/2900] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:37:26.826346 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d0f5e-0108-4268-bf61-f728994c1486 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:26.827115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d0f5e-0108-4268-bf61-f728994c1486 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:26.827115 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d0f5e-0108-4268-bf61-f728994c1486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:26.849245 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e59d0f5e-0108-4268-bf61-f728994c1486 None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:37:26.850482 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e59d0f5e-0108-4268-bf61-f728994c1486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:26.859872 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1462/2901] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:37:26.864332 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa57894-aae4-43ca-84e3-60a7b42da7ab None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:26.874676 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa57894-aae4-43ca-84e3-60a7b42da7ab None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:26.879511 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:26.880110 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:26.880382 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:26.880648 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:26.881340 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81607) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 08:37:26.881665 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:26.883304 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:26.885025 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:26.897099 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1440/2902] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Aug 23 08:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 08:37:26.903230 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d0f5e-0108-4268-bf61-f728994c1486 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:26.914075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d0f5e-0108-4268-bf61-f728994c1486 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:26.919613 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db4ee7-af72-43f9-851e-79c501976bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:26.920272 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4ee7-af72-43f9-851e-79c501976bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:26.920575 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4ee7-af72-43f9-851e-79c501976bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:26.920846 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4ee7-af72-43f9-851e-79c501976bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:26.921595 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db4ee7-af72-43f9-851e-79c501976bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:26.923620 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db4ee7-af72-43f9-851e-79c501976bf3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:27.011009 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1463/2903] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Aug 23 08:37:26 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:37:27.015222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:27.025560 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9132359-92f5-45b5-bd57-1cb674310fd4 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:27.086586 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e654c-634d-4208-a704-8036c36e9522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:27.087222 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e654c-634d-4208-a704-8036c36e9522 admin admin] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:27.087495 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e654c-634d-4208-a704-8036c36e9522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:27.087761 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e654c-634d-4208-a704-8036c36e9522 admin admin] PATH_INFO: `/v3/users` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:27.088390 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e654c-634d-4208-a704-8036c36e9522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:27.090323 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e654c-634d-4208-a704-8036c36e9522 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:27.203731 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1441/2904] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Aug 23 08:37:27 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 08:37:27.208377 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db4ee7-af72-43f9-851e-79c501976bf3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:27.219898 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db4ee7-af72-43f9-851e-79c501976bf3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:27.225330 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23f26a-0ad5-434f-a5e6-46f09c0fa103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:27.225958 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23f26a-0ad5-434f-a5e6-46f09c0fa103 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:27.226259 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23f26a-0ad5-434f-a5e6-46f09c0fa103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:27.226558 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23f26a-0ad5-434f-a5e6-46f09c0fa103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:27.227292 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23f26a-0ad5-434f-a5e6-46f09c0fa103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:27.228972 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23f26a-0ad5-434f-a5e6-46f09c0fa103 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:27.240914 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1464/2905] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:37:27.245259 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e654c-634d-4208-a704-8036c36e9522 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:27.256003 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e654c-634d-4208-a704-8036c36e9522 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:27.260852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f947f-45c5-4d8f-8afd-65a8fd14b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:27.261506 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f947f-45c5-4d8f-8afd-65a8fd14b9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:27.261748 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f947f-45c5-4d8f-8afd-65a8fd14b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:27.262015 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f947f-45c5-4d8f-8afd-65a8fd14b9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:27.262662 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f947f-45c5-4d8f-8afd-65a8fd14b9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:27.264312 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f947f-45c5-4d8f-8afd-65a8fd14b9b4 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:27.275937 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1442/2906] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Aug 23 08:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 08:37:27.281045 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23f26a-0ad5-434f-a5e6-46f09c0fa103 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:27.291592 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23f26a-0ad5-434f-a5e6-46f09c0fa103 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:27.296913 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:27.297576 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:27.297856 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:27.298129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] PATH_INFO: `/v3/projects/59da7836125b4697948f46e76e52b0bb/users/993eed9ed5b94add9606263f2f723318/roles/3b325dcac3064617a5415a84759ef71a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:27.313887 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59da7836125b4697948f46e76e52b0bb, user_id=993eed9ed5b94add9606263f2f723318, role_id=3b325dcac3064617a5415a84759ef71a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:27.316606 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:27.392936 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1465/2907] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Aug 23 08:37:27 2022] PUT /identity/v3/projects/59da7836125b4697948f46e76e52b0bb/users/993eed9ed5b94add9606263f2f723318/roles/3b325dcac3064617a5415a84759ef71a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:37:27.403852 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076cbc0-31a5-4239-9e14-baaf87e4af1f None None] REQUEST_METHOD: `POST` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:27.404137 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076cbc0-31a5-4239-9e14-baaf87e4af1f None None] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:27.404416 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076cbc0-31a5-4239-9e14-baaf87e4af1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:27.439782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.auth.core [None req-b076cbc0-31a5-4239-9e14-baaf87e4af1f None None] MFA Rules not processed for user `993eed9ed5b94add9606263f2f723318`. Rule list: `[]` (Enabled: `True`). {{(pid=81607) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:37:27.441481 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-b076cbc0-31a5-4239-9e14-baaf87e4af1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:27.518913 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1443/2908] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:37:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:37:27.530572 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:27.532297 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:27.543515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:27.593384 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088baa9-d9fd-430a-bb69-e208492bc027 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:27.597053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34722f03-f67c-4b7c-ade4-95d3b7bab37c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6adffd03ce3a4ebfb2d7c5df3a2ca293', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:27.597669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722f03-f67c-4b7c-ade4-95d3b7bab37c service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:27.597935 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722f03-f67c-4b7c-ade4-95d3b7bab37c service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:27.598204 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34722f03-f67c-4b7c-ade4-95d3b7bab37c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:27.607509 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34722f03-f67c-4b7c-ade4-95d3b7bab37c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:27.608967 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34722f03-f67c-4b7c-ade4-95d3b7bab37c service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:27.619386 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1466/2909] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:37:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:37:37.511113 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:37.745739 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=22 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:37:53.073230 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076cbc0-31a5-4239-9e14-baaf87e4af1f None None] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:53.148182 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076cbc0-31a5-4239-9e14-baaf87e4af1f None None] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:53.155413 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:53.156138 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:53.156463 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:53.156782 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] PATH_INFO: `/v3/users/993eed9ed5b94add9606263f2f723318` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:53.158116 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] RBAC: Authorizing `identity:delete_user(user_id=993eed9ed5b94add9606263f2f723318)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:53.160419 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:53.395446 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993eed9ed5b94add9606263f2f723318'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:53.396027 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993eed9ed5b94add9606263f2f723318'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:53.403480 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993eed9ed5b94add9606263f2f723318'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:53.404129 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993eed9ed5b94add9606263f2f723318'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:53.481693 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993eed9ed5b94add9606263f2f723318'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:53.500986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1444/2910] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:37:53 2022] DELETE /identity/v3/users/993eed9ed5b94add9606263f2f723318 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 08:37:53.506669 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d806-a6f0-4fcf-aa42-666052e28ecb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:53.507215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d806-a6f0-4fcf-aa42-666052e28ecb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:53.507402 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d806-a6f0-4fcf-aa42-666052e28ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:53.532069 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d8a6d806-a6f0-4fcf-aa42-666052e28ecb None None] MFA Rules not processed for user `70cd347e5af341fcb03bb3178f2c4247`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 08:37:53.533579 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8a6d806-a6f0-4fcf-aa42-666052e28ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:53.599746 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1467/2911] 10.222.0.58 () {58 vars in 958 bytes} [Tue Aug 23 08:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 08:37:53.609935 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:53.611411 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:53.669202 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0bb853-676b-4117-8e63-63e89d74b21f admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:53.672907 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff58d67-e6aa-4971-8282-4a028d67b5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:53.673509 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff58d67-e6aa-4971-8282-4a028d67b5ca service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:53.673786 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff58d67-e6aa-4971-8282-4a028d67b5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:53.674054 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff58d67-e6aa-4971-8282-4a028d67b5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:53.683749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff58d67-e6aa-4971-8282-4a028d67b5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:53.685174 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff58d67-e6aa-4971-8282-4a028d67b5ca service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:53.694986 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1445/2912] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:37:54.147641 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6d806-a6f0-4fcf-aa42-666052e28ecb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:37:54.149118 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8a6d806-a6f0-4fcf-aa42-666052e28ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:37:54.164169 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6d806-a6f0-4fcf-aa42-666052e28ecb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:37:54.169075 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:37:54.169728 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:37:54.170034 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:37:54.170327 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] PATH_INFO: `/v3/projects/59da7836125b4697948f46e76e52b0bb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:37:54.171507 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59da7836125b4697948f46e76e52b0bb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:37:54.173490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:37:54.308014 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59da7836125b4697948f46e76e52b0bb', 'request_id': 'req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:54.409513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59da7836125b4697948f46e76e52b0bb', 'request_id': 'req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:54.416516 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59da7836125b4697948f46e76e52b0bb', 'request_id': 'req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:54.428512 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] The token cache is being invalidate because project 59da7836125b4697948f46e76e52b0bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:37:54.428804 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:54.429504 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:37:54.431015 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1468/2913] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:37:54 2022] DELETE /identity/v3/projects/59da7836125b4697948f46e76e52b0bb => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:38:03.690660 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:38:04.425491 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:38:28.920740 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff58d67-e6aa-4971-8282-4a028d67b5ca service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:38:28.929558 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff58d67-e6aa-4971-8282-4a028d67b5ca service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:38:28.984674 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:38:28.985286 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:38:28.985561 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:38:28.985827 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:38:28.987640 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:38:28.997632 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:38:28.999807 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:38:29.008493 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1446/2914] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:38:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:38:39.004038 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=26 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:39:22.964682 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:39:22.968260 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:23.094453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3ac0b-7305-4131-a489-c7eb4efb3f70 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:39:23.100467 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03cc4435b9b0444b9f278e15398443c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:39:23.101074 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:39:23.101394 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:39:23.101667 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:39:23.103490 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:23.118815 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:39:23.120631 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:39:23.132053 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1469/2915] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:39:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:39:23.670114 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:39:23.671717 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:23.682855 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e960d20-ea36-4340-ad64-36676cafdc42 service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:39:23.686838 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:39:23.687476 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:39:23.687757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:39:23.688026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:39:23.690407 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:23.704564 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:39:23.706002 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:39:23.718490 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1447/2916] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:39:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:39:24.913154 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:39:24.923310 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611aa9-18db-4308-aed3-a7d9052555dc service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:39:24.927646 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:39:24.928261 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:39:24.928552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:39:24.928834 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:39:24.930759 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:24.946453 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:39:24.948129 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:39:24.959567 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1470/2917] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:39:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:39:33.712452 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=6 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:39:34.954304 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=52 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:39:38.504492 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:39:38.514256 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0d521-5e35-461e-9af0-fb0eef33740a service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:39:38.518131 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd99632164b3b4a1f95736a65da217db2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:39:38.518766 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] REQUEST_METHOD: `GET` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:39:38.519080 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:39:38.519354 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:39:38.521835 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:38.595548 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:39:38.597154 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:39:38.609216 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1448/2918] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:39:46.515266 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:39:46.516933 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:46.538055 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5031b9-3b79-4b21-bd60-038b5aa8518c service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:39:46.542973 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:39:46.543623 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:39:46.543899 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:39:46.544167 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] PATH_INFO: `/v3/users/db8b5a8edcc34d32a3e9fad54d33cbbc` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:39:46.545513 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8b5a8edcc34d32a3e9fad54d33cbbc)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:39:46.547444 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:39:46.790748 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8b5a8edcc34d32a3e9fad54d33cbbc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:46.797850 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8b5a8edcc34d32a3e9fad54d33cbbc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:46.798550 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8b5a8edcc34d32a3e9fad54d33cbbc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:46.799208 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8b5a8edcc34d32a3e9fad54d33cbbc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:46.812734 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8b5a8edcc34d32a3e9fad54d33cbbc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:46.901652 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1471/2919] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:39:46 2022] DELETE /identity/v3/users/db8b5a8edcc34d32a3e9fad54d33cbbc => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:39:47.188589 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:39:47.198909 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3e63a-3566-4567-97a2-fc3b8d54015c service neutron] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:39:47.262757 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:39:47.263469 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:39:47.263750 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:39:47.264026 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] PATH_INFO: `/v3/projects/7625c9efcc924680a5efef3fd71d0272` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:39:47.265166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7625c9efcc924680a5efef3fd71d0272)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:39:47.267297 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:39:47.388749 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7625c9efcc924680a5efef3fd71d0272', 'request_id': 'req-17f559f4-0abc-41be-809e-dcfa55490837'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.395058 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7625c9efcc924680a5efef3fd71d0272', 'request_id': 'req-17f559f4-0abc-41be-809e-dcfa55490837'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.401514 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7625c9efcc924680a5efef3fd71d0272', 'request_id': 'req-17f559f4-0abc-41be-809e-dcfa55490837'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.509499 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] The token cache is being invalidate because project 7625c9efcc924680a5efef3fd71d0272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:39:47.509804 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.510410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.511958 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1449/2920] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:39:47 2022] DELETE /identity/v3/projects/7625c9efcc924680a5efef3fd71d0272 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:39:47.516231 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:39:47.517837 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:47.533574 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1d29f-ca17-4456-bc8f-6113bc8f9faa admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:39:47.594070 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:39:47.594701 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:39:47.595016 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:39:47.595293 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] PATH_INFO: `/v3/users/c113f1533907487e81ddb547c31b26dd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:39:47.596515 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c113f1533907487e81ddb547c31b26dd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:39:47.599219 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:39:47.872474 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c113f1533907487e81ddb547c31b26dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.880215 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c113f1533907487e81ddb547c31b26dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.880940 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c113f1533907487e81ddb547c31b26dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.881552 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c113f1533907487e81ddb547c31b26dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.896229 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c113f1533907487e81ddb547c31b26dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:47.898252 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:39:48.004625 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1472/2921] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Aug 23 08:39:47 2022] DELETE /identity/v3/users/c113f1533907487e81ddb547c31b26dd => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:39:48.263170 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] Authenticating user token {{(pid=81607) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:39:48.267410 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.fernet_utils [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81607) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:39:48.278108 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f559f4-0abc-41be-809e-dcfa55490837 admin admin] Validating token access rules against request {{(pid=81607) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:39:48.301037 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:39:48.338709 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cd347e5af341fcb03bb3178f2c4247', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee2616c3f460471aaf732aeea2d3bd15', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81607) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:39:48.339377 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:39:48.339657 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:39:48.339980 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] PATH_INFO: `/v3/projects/21c42631bb974a16a633bc24b885dd41` {{(pid=81607) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:39:48.341152 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c42631bb974a16a633bc24b885dd41)` {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:39:48.343098 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] RBAC: Authorization granted {{(pid=81607) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:39:48.513166 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c42631bb974a16a633bc24b885dd41', 'request_id': 'req-9f548939-ba19-4bae-ad7b-5275332b9b2d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:48.518535 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c42631bb974a16a633bc24b885dd41', 'request_id': 'req-9f548939-ba19-4bae-ad7b-5275332b9b2d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:48.524978 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c42631bb974a16a633bc24b885dd41', 'request_id': 'req-9f548939-ba19-4bae-ad7b-5275332b9b2d'} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:48.623578 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] The token cache is being invalidate because project 21c42631bb974a16a633bc24b885dd41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81607) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 08:39:48.623814 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:48.624402 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG keystone.notifications [None req-9f548939-ba19-4bae-ad7b-5275332b9b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81607) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 08:39:48.625910 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: [pid: 81607|app: 0|req: 1450/2922] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Aug 23 08:39:48 2022] DELETE /identity/v3/projects/21c42631bb974a16a633bc24b885dd41 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 08:39:58.152661 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:39:58.795513 dv-cinder-854070-4-iscsi devstack@keystone.service[81607]: DEBUG dbcounter [-] [81607] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81607) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 08:41:24.176184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 08:41:24.177609 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:41:24.251527 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72b705-4139-490d-a257-e45a45d1bba8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 08:41:24.312839 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec579b55-f7c5-43c7-9552-8254b2f35e14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40df52dd29044cddbd521b59dd30b6a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1c63553f4ca4c1eacf87634168afe18', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 08:41:24.314051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec579b55-f7c5-43c7-9552-8254b2f35e14 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 08:41:24.314051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec579b55-f7c5-43c7-9552-8254b2f35e14 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 08:41:24.314051 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec579b55-f7c5-43c7-9552-8254b2f35e14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 08:41:24.316426 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ec579b55-f7c5-43c7-9552-8254b2f35e14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 08:41:24.328193 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec579b55-f7c5-43c7-9552-8254b2f35e14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 08:41:24.329787 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec579b55-f7c5-43c7-9552-8254b2f35e14 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 08:41:24.340184 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1473/2923] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Aug 23 08:41:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 08:41:34.335826 dv-cinder-854070-4-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=58 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}